1*acda97acSIgor Matheus Andrade Torrente======================= 2*acda97acSIgor Matheus Andrade TorrenteDirect Access for files 3*acda97acSIgor Matheus Andrade Torrente======================= 4*acda97acSIgor Matheus Andrade Torrente 5*acda97acSIgor Matheus Andrade TorrenteMotivation 6*acda97acSIgor Matheus Andrade Torrente---------- 7*acda97acSIgor Matheus Andrade Torrente 8*acda97acSIgor Matheus Andrade TorrenteThe page cache is usually used to buffer reads and writes to files. 9*acda97acSIgor Matheus Andrade TorrenteIt is also used to provide the pages which are mapped into userspace 10*acda97acSIgor Matheus Andrade Torrenteby a call to mmap. 11*acda97acSIgor Matheus Andrade Torrente 12*acda97acSIgor Matheus Andrade TorrenteFor block devices that are memory-like, the page cache pages would be 13*acda97acSIgor Matheus Andrade Torrenteunnecessary copies of the original storage. The `DAX` code removes the 14*acda97acSIgor Matheus Andrade Torrenteextra copy by performing reads and writes directly to the storage device. 15*acda97acSIgor Matheus Andrade TorrenteFor file mappings, the storage device is mapped directly into userspace. 16*acda97acSIgor Matheus Andrade Torrente 17*acda97acSIgor Matheus Andrade Torrente 18*acda97acSIgor Matheus Andrade TorrenteUsage 19*acda97acSIgor Matheus Andrade Torrente----- 20*acda97acSIgor Matheus Andrade Torrente 21*acda97acSIgor Matheus Andrade TorrenteIf you have a block device which supports `DAX`, you can make a filesystem 22*acda97acSIgor Matheus Andrade Torrenteon it as usual. The `DAX` code currently only supports files with a block 23*acda97acSIgor Matheus Andrade Torrentesize equal to your kernel's `PAGE_SIZE`, so you may need to specify a block 24*acda97acSIgor Matheus Andrade Torrentesize when creating the filesystem. 25*acda97acSIgor Matheus Andrade Torrente 26*acda97acSIgor Matheus Andrade TorrenteCurrently 3 filesystems support `DAX`: ext2, ext4 and xfs. Enabling `DAX` on them 27*acda97acSIgor Matheus Andrade Torrenteis different. 28*acda97acSIgor Matheus Andrade Torrente 29*acda97acSIgor Matheus Andrade TorrenteEnabling DAX on ext2 30*acda97acSIgor Matheus Andrade Torrente-------------------- 31*acda97acSIgor Matheus Andrade Torrente 32*acda97acSIgor Matheus Andrade TorrenteWhen mounting the filesystem, use the ``-o dax`` option on the command line or 33*acda97acSIgor Matheus Andrade Torrenteadd 'dax' to the options in ``/etc/fstab``. This works to enable `DAX` on all files 34*acda97acSIgor Matheus Andrade Torrentewithin the filesystem. It is equivalent to the ``-o dax=always`` behavior below. 35*acda97acSIgor Matheus Andrade Torrente 36*acda97acSIgor Matheus Andrade Torrente 37*acda97acSIgor Matheus Andrade TorrenteEnabling DAX on xfs and ext4 38*acda97acSIgor Matheus Andrade Torrente---------------------------- 39*acda97acSIgor Matheus Andrade Torrente 40*acda97acSIgor Matheus Andrade TorrenteSummary 41*acda97acSIgor Matheus Andrade Torrente------- 42*acda97acSIgor Matheus Andrade Torrente 43*acda97acSIgor Matheus Andrade Torrente 1. There exists an in-kernel file access mode flag `S_DAX` that corresponds to 44*acda97acSIgor Matheus Andrade Torrente the statx flag `STATX_ATTR_DAX`. See the manpage for statx(2) for details 45*acda97acSIgor Matheus Andrade Torrente about this access mode. 46*acda97acSIgor Matheus Andrade Torrente 47*acda97acSIgor Matheus Andrade Torrente 2. There exists a persistent flag `FS_XFLAG_DAX` that can be applied to regular 48*acda97acSIgor Matheus Andrade Torrente files and directories. This advisory flag can be set or cleared at any 49*acda97acSIgor Matheus Andrade Torrente time, but doing so does not immediately affect the `S_DAX` state. 50*acda97acSIgor Matheus Andrade Torrente 51*acda97acSIgor Matheus Andrade Torrente 3. If the persistent `FS_XFLAG_DAX` flag is set on a directory, this flag will 52*acda97acSIgor Matheus Andrade Torrente be inherited by all regular files and subdirectories that are subsequently 53*acda97acSIgor Matheus Andrade Torrente created in this directory. Files and subdirectories that exist at the time 54*acda97acSIgor Matheus Andrade Torrente this flag is set or cleared on the parent directory are not modified by 55*acda97acSIgor Matheus Andrade Torrente this modification of the parent directory. 56*acda97acSIgor Matheus Andrade Torrente 57*acda97acSIgor Matheus Andrade Torrente 4. There exist dax mount options which can override `FS_XFLAG_DAX` in the 58*acda97acSIgor Matheus Andrade Torrente setting of the `S_DAX` flag. Given underlying storage which supports `DAX` the 59*acda97acSIgor Matheus Andrade Torrente following hold: 60*acda97acSIgor Matheus Andrade Torrente 61*acda97acSIgor Matheus Andrade Torrente ``-o dax=inode`` means "follow `FS_XFLAG_DAX`" and is the default. 62*acda97acSIgor Matheus Andrade Torrente 63*acda97acSIgor Matheus Andrade Torrente ``-o dax=never`` means "never set `S_DAX`, ignore `FS_XFLAG_DAX`." 64*acda97acSIgor Matheus Andrade Torrente 65*acda97acSIgor Matheus Andrade Torrente ``-o dax=always`` means "always set `S_DAX` ignore `FS_XFLAG_DAX`." 66*acda97acSIgor Matheus Andrade Torrente 67*acda97acSIgor Matheus Andrade Torrente ``-o dax`` is a legacy option which is an alias for ``dax=always``. 68*acda97acSIgor Matheus Andrade Torrente 69*acda97acSIgor Matheus Andrade Torrente .. warning:: 70*acda97acSIgor Matheus Andrade Torrente 71*acda97acSIgor Matheus Andrade Torrente The option ``-o dax`` may be removed in the future so ``-o dax=always`` is 72*acda97acSIgor Matheus Andrade Torrente the preferred method for specifying this behavior. 73*acda97acSIgor Matheus Andrade Torrente 74*acda97acSIgor Matheus Andrade Torrente .. note:: 75*acda97acSIgor Matheus Andrade Torrente 76*acda97acSIgor Matheus Andrade Torrente Modifications to and the inheritance behavior of `FS_XFLAG_DAX` remain 77*acda97acSIgor Matheus Andrade Torrente the same even when the filesystem is mounted with a dax option. However, 78*acda97acSIgor Matheus Andrade Torrente in-core inode state (`S_DAX`) will be overridden until the filesystem is 79*acda97acSIgor Matheus Andrade Torrente remounted with dax=inode and the inode is evicted from kernel memory. 80*acda97acSIgor Matheus Andrade Torrente 81*acda97acSIgor Matheus Andrade Torrente 5. The `S_DAX` policy can be changed via: 82*acda97acSIgor Matheus Andrade Torrente 83*acda97acSIgor Matheus Andrade Torrente a) Setting the parent directory `FS_XFLAG_DAX` as needed before files are 84*acda97acSIgor Matheus Andrade Torrente created 85*acda97acSIgor Matheus Andrade Torrente 86*acda97acSIgor Matheus Andrade Torrente b) Setting the appropriate dax="foo" mount option 87*acda97acSIgor Matheus Andrade Torrente 88*acda97acSIgor Matheus Andrade Torrente c) Changing the `FS_XFLAG_DAX` flag on existing regular files and 89*acda97acSIgor Matheus Andrade Torrente directories. This has runtime constraints and limitations that are 90*acda97acSIgor Matheus Andrade Torrente described in 6) below. 91*acda97acSIgor Matheus Andrade Torrente 92*acda97acSIgor Matheus Andrade Torrente 6. When changing the `S_DAX` policy via toggling the persistent `FS_XFLAG_DAX` 93*acda97acSIgor Matheus Andrade Torrente flag, the change to existing regular files won't take effect until the 94*acda97acSIgor Matheus Andrade Torrente files are closed by all processes. 95*acda97acSIgor Matheus Andrade Torrente 96*acda97acSIgor Matheus Andrade Torrente 97*acda97acSIgor Matheus Andrade TorrenteDetails 98*acda97acSIgor Matheus Andrade Torrente------- 99*acda97acSIgor Matheus Andrade Torrente 100*acda97acSIgor Matheus Andrade TorrenteThere are 2 per-file dax flags. One is a persistent inode setting (`FS_XFLAG_DAX`) 101*acda97acSIgor Matheus Andrade Torrenteand the other is a volatile flag indicating the active state of the feature 102*acda97acSIgor Matheus Andrade Torrente(`S_DAX`). 103*acda97acSIgor Matheus Andrade Torrente 104*acda97acSIgor Matheus Andrade Torrente`FS_XFLAG_DAX` is preserved within the filesystem. This persistent config 105*acda97acSIgor Matheus Andrade Torrentesetting can be set, cleared and/or queried using the `FS_IOC_FS`[`GS`]`ETXATTR` ioctl 106*acda97acSIgor Matheus Andrade Torrente(see ioctl_xfs_fsgetxattr(2)) or an utility such as 'xfs_io'. 107*acda97acSIgor Matheus Andrade Torrente 108*acda97acSIgor Matheus Andrade TorrenteNew files and directories automatically inherit `FS_XFLAG_DAX` from 109*acda97acSIgor Matheus Andrade Torrentetheir parent directory **when created**. Therefore, setting `FS_XFLAG_DAX` at 110*acda97acSIgor Matheus Andrade Torrentedirectory creation time can be used to set a default behavior for an entire 111*acda97acSIgor Matheus Andrade Torrentesub-tree. 112*acda97acSIgor Matheus Andrade Torrente 113*acda97acSIgor Matheus Andrade TorrenteTo clarify inheritance, here are 3 examples: 114*acda97acSIgor Matheus Andrade Torrente 115*acda97acSIgor Matheus Andrade TorrenteExample A: 116*acda97acSIgor Matheus Andrade Torrente 117*acda97acSIgor Matheus Andrade Torrente.. code-block:: shell 118*acda97acSIgor Matheus Andrade Torrente 119*acda97acSIgor Matheus Andrade Torrente mkdir -p a/b/c 120*acda97acSIgor Matheus Andrade Torrente xfs_io -c 'chattr +x' a 121*acda97acSIgor Matheus Andrade Torrente mkdir a/b/c/d 122*acda97acSIgor Matheus Andrade Torrente mkdir a/e 123*acda97acSIgor Matheus Andrade Torrente 124*acda97acSIgor Matheus Andrade Torrente ------[outcome]------ 125*acda97acSIgor Matheus Andrade Torrente 126*acda97acSIgor Matheus Andrade Torrente dax: a,e 127*acda97acSIgor Matheus Andrade Torrente no dax: b,c,d 128*acda97acSIgor Matheus Andrade Torrente 129*acda97acSIgor Matheus Andrade TorrenteExample B: 130*acda97acSIgor Matheus Andrade Torrente 131*acda97acSIgor Matheus Andrade Torrente.. code-block:: shell 132*acda97acSIgor Matheus Andrade Torrente 133*acda97acSIgor Matheus Andrade Torrente mkdir a 134*acda97acSIgor Matheus Andrade Torrente xfs_io -c 'chattr +x' a 135*acda97acSIgor Matheus Andrade Torrente mkdir -p a/b/c/d 136*acda97acSIgor Matheus Andrade Torrente 137*acda97acSIgor Matheus Andrade Torrente ------[outcome]------ 138*acda97acSIgor Matheus Andrade Torrente 139*acda97acSIgor Matheus Andrade Torrente dax: a,b,c,d 140*acda97acSIgor Matheus Andrade Torrente no dax: 141*acda97acSIgor Matheus Andrade Torrente 142*acda97acSIgor Matheus Andrade TorrenteExample C: 143*acda97acSIgor Matheus Andrade Torrente 144*acda97acSIgor Matheus Andrade Torrente.. code-block:: shell 145*acda97acSIgor Matheus Andrade Torrente 146*acda97acSIgor Matheus Andrade Torrente mkdir -p a/b/c 147*acda97acSIgor Matheus Andrade Torrente xfs_io -c 'chattr +x' c 148*acda97acSIgor Matheus Andrade Torrente mkdir a/b/c/d 149*acda97acSIgor Matheus Andrade Torrente 150*acda97acSIgor Matheus Andrade Torrente ------[outcome]------ 151*acda97acSIgor Matheus Andrade Torrente 152*acda97acSIgor Matheus Andrade Torrente dax: c,d 153*acda97acSIgor Matheus Andrade Torrente no dax: a,b 154*acda97acSIgor Matheus Andrade Torrente 155*acda97acSIgor Matheus Andrade TorrenteThe current enabled state (`S_DAX`) is set when a file inode is instantiated in 156*acda97acSIgor Matheus Andrade Torrentememory by the kernel. It is set based on the underlying media support, the 157*acda97acSIgor Matheus Andrade Torrentevalue of `FS_XFLAG_DAX` and the filesystem's dax mount option. 158*acda97acSIgor Matheus Andrade Torrente 159*acda97acSIgor Matheus Andrade Torrentestatx can be used to query `S_DAX`. 160*acda97acSIgor Matheus Andrade Torrente 161*acda97acSIgor Matheus Andrade Torrente.. note:: 162*acda97acSIgor Matheus Andrade Torrente 163*acda97acSIgor Matheus Andrade Torrente That only regular files will ever have `S_DAX` set and therefore statx 164*acda97acSIgor Matheus Andrade Torrente will never indicate that `S_DAX` is set on directories. 165*acda97acSIgor Matheus Andrade Torrente 166*acda97acSIgor Matheus Andrade TorrenteSetting the `FS_XFLAG_DAX` flag (specifically or through inheritance) occurs even 167*acda97acSIgor Matheus Andrade Torrenteif the underlying media does not support dax and/or the filesystem is 168*acda97acSIgor Matheus Andrade Torrenteoverridden with a mount option. 169*acda97acSIgor Matheus Andrade Torrente 170*acda97acSIgor Matheus Andrade Torrente 171*acda97acSIgor Matheus Andrade TorrenteImplementation Tips for Block Driver Writers 172*acda97acSIgor Matheus Andrade Torrente-------------------------------------------- 173*acda97acSIgor Matheus Andrade Torrente 174*acda97acSIgor Matheus Andrade TorrenteTo support `DAX` in your block driver, implement the 'direct_access' 175*acda97acSIgor Matheus Andrade Torrenteblock device operation. It is used to translate the sector number 176*acda97acSIgor Matheus Andrade Torrente(expressed in units of 512-byte sectors) to a page frame number (pfn) 177*acda97acSIgor Matheus Andrade Torrentethat identifies the physical page for the memory. It also returns a 178*acda97acSIgor Matheus Andrade Torrentekernel virtual address that can be used to access the memory. 179*acda97acSIgor Matheus Andrade Torrente 180*acda97acSIgor Matheus Andrade TorrenteThe direct_access method takes a 'size' parameter that indicates the 181*acda97acSIgor Matheus Andrade Torrentenumber of bytes being requested. The function should return the number 182*acda97acSIgor Matheus Andrade Torrenteof bytes that can be contiguously accessed at that offset. It may also 183*acda97acSIgor Matheus Andrade Torrentereturn a negative errno if an error occurs. 184*acda97acSIgor Matheus Andrade Torrente 185*acda97acSIgor Matheus Andrade TorrenteIn order to support this method, the storage must be byte-accessible by 186*acda97acSIgor Matheus Andrade Torrentethe CPU at all times. If your device uses paging techniques to expose 187*acda97acSIgor Matheus Andrade Torrentea large amount of memory through a smaller window, then you cannot 188*acda97acSIgor Matheus Andrade Torrenteimplement direct_access. Equally, if your device can occasionally 189*acda97acSIgor Matheus Andrade Torrentestall the CPU for an extended period, you should also not attempt to 190*acda97acSIgor Matheus Andrade Torrenteimplement direct_access. 191*acda97acSIgor Matheus Andrade Torrente 192*acda97acSIgor Matheus Andrade TorrenteThese block devices may be used for inspiration: 193*acda97acSIgor Matheus Andrade Torrente- brd: RAM backed block device driver 194*acda97acSIgor Matheus Andrade Torrente- dcssblk: s390 dcss block device driver 195*acda97acSIgor Matheus Andrade Torrente- pmem: NVDIMM persistent memory driver 196*acda97acSIgor Matheus Andrade Torrente 197*acda97acSIgor Matheus Andrade Torrente 198*acda97acSIgor Matheus Andrade TorrenteImplementation Tips for Filesystem Writers 199*acda97acSIgor Matheus Andrade Torrente------------------------------------------ 200*acda97acSIgor Matheus Andrade Torrente 201*acda97acSIgor Matheus Andrade TorrenteFilesystem support consists of: 202*acda97acSIgor Matheus Andrade Torrente 203*acda97acSIgor Matheus Andrade Torrente* Adding support to mark inodes as being `DAX` by setting the `S_DAX` flag in 204*acda97acSIgor Matheus Andrade Torrente i_flags 205*acda97acSIgor Matheus Andrade Torrente* Implementing ->read_iter and ->write_iter operations which use 206*acda97acSIgor Matheus Andrade Torrente :c:func:`dax_iomap_rw()` when inode has `S_DAX` flag set 207*acda97acSIgor Matheus Andrade Torrente* Implementing an mmap file operation for `DAX` files which sets the 208*acda97acSIgor Matheus Andrade Torrente `VM_MIXEDMAP` and `VM_HUGEPAGE` flags on the `VMA`, and setting the vm_ops to 209*acda97acSIgor Matheus Andrade Torrente include handlers for fault, pmd_fault, page_mkwrite, pfn_mkwrite. These 210*acda97acSIgor Matheus Andrade Torrente handlers should probably call :c:func:`dax_iomap_fault()` passing the 211*acda97acSIgor Matheus Andrade Torrente appropriate fault size and iomap operations. 212*acda97acSIgor Matheus Andrade Torrente* Calling :c:func:`iomap_zero_range()` passing appropriate iomap operations 213*acda97acSIgor Matheus Andrade Torrente instead of :c:func:`block_truncate_page()` for `DAX` files 214*acda97acSIgor Matheus Andrade Torrente* Ensuring that there is sufficient locking between reads, writes, 215*acda97acSIgor Matheus Andrade Torrente truncates and page faults 216*acda97acSIgor Matheus Andrade Torrente 217*acda97acSIgor Matheus Andrade TorrenteThe iomap handlers for allocating blocks must make sure that allocated blocks 218*acda97acSIgor Matheus Andrade Torrenteare zeroed out and converted to written extents before being returned to avoid 219*acda97acSIgor Matheus Andrade Torrenteexposure of uninitialized data through mmap. 220*acda97acSIgor Matheus Andrade Torrente 221*acda97acSIgor Matheus Andrade TorrenteThese filesystems may be used for inspiration: 222*acda97acSIgor Matheus Andrade Torrente 223*acda97acSIgor Matheus Andrade Torrente.. seealso:: 224*acda97acSIgor Matheus Andrade Torrente 225*acda97acSIgor Matheus Andrade Torrente ext2: see Documentation/filesystems/ext2.rst 226*acda97acSIgor Matheus Andrade Torrente 227*acda97acSIgor Matheus Andrade Torrente.. seealso:: 228*acda97acSIgor Matheus Andrade Torrente 229*acda97acSIgor Matheus Andrade Torrente xfs: see Documentation/admin-guide/xfs.rst 230*acda97acSIgor Matheus Andrade Torrente 231*acda97acSIgor Matheus Andrade Torrente.. seealso:: 232*acda97acSIgor Matheus Andrade Torrente 233*acda97acSIgor Matheus Andrade Torrente ext4: see Documentation/filesystems/ext4/ 234*acda97acSIgor Matheus Andrade Torrente 235*acda97acSIgor Matheus Andrade Torrente 236*acda97acSIgor Matheus Andrade TorrenteHandling Media Errors 237*acda97acSIgor Matheus Andrade Torrente--------------------- 238*acda97acSIgor Matheus Andrade Torrente 239*acda97acSIgor Matheus Andrade TorrenteThe libnvdimm subsystem stores a record of known media error locations for 240*acda97acSIgor Matheus Andrade Torrenteeach pmem block device (in gendisk->badblocks). If we fault at such location, 241*acda97acSIgor Matheus Andrade Torrenteor one with a latent error not yet discovered, the application can expect 242*acda97acSIgor Matheus Andrade Torrenteto receive a `SIGBUS`. Libnvdimm also allows clearing of these errors by simply 243*acda97acSIgor Matheus Andrade Torrentewriting the affected sectors (through the pmem driver, and if the underlying 244*acda97acSIgor Matheus Andrade TorrenteNVDIMM supports the clear_poison DSM defined by ACPI). 245*acda97acSIgor Matheus Andrade Torrente 246*acda97acSIgor Matheus Andrade TorrenteSince `DAX` IO normally doesn't go through the ``driver/bio`` path, applications or 247*acda97acSIgor Matheus Andrade Torrentesysadmins have an option to restore the lost data from a prior ``backup/inbuilt`` 248*acda97acSIgor Matheus Andrade Torrenteredundancy in the following ways: 249*acda97acSIgor Matheus Andrade Torrente 250*acda97acSIgor Matheus Andrade Torrente1. Delete the affected file, and restore from a backup (sysadmin route): 251*acda97acSIgor Matheus Andrade Torrente This will free the filesystem blocks that were being used by the file, 252*acda97acSIgor Matheus Andrade Torrente and the next time they're allocated, they will be zeroed first, which 253*acda97acSIgor Matheus Andrade Torrente happens through the driver, and will clear bad sectors. 254*acda97acSIgor Matheus Andrade Torrente 255*acda97acSIgor Matheus Andrade Torrente2. Truncate or hole-punch the part of the file that has a bad-block (at least 256*acda97acSIgor Matheus Andrade Torrente an entire aligned sector has to be hole-punched, but not necessarily an 257*acda97acSIgor Matheus Andrade Torrente entire filesystem block). 258*acda97acSIgor Matheus Andrade Torrente 259*acda97acSIgor Matheus Andrade TorrenteThese are the two basic paths that allow `DAX` filesystems to continue operating 260*acda97acSIgor Matheus Andrade Torrentein the presence of media errors. More robust error recovery mechanisms can be 261*acda97acSIgor Matheus Andrade Torrentebuilt on top of this in the future, for example, involving redundancy/mirroring 262*acda97acSIgor Matheus Andrade Torrenteprovided at the block layer through DM, or additionally, at the filesystem 263*acda97acSIgor Matheus Andrade Torrentelevel. These would have to rely on the above two tenets, that error clearing 264*acda97acSIgor Matheus Andrade Torrentecan happen either by sending an IO through the driver, or zeroing (also through 265*acda97acSIgor Matheus Andrade Torrentethe driver). 266*acda97acSIgor Matheus Andrade Torrente 267*acda97acSIgor Matheus Andrade Torrente 268*acda97acSIgor Matheus Andrade TorrenteShortcomings 269*acda97acSIgor Matheus Andrade Torrente------------ 270*acda97acSIgor Matheus Andrade Torrente 271*acda97acSIgor Matheus Andrade TorrenteEven if the kernel or its modules are stored on a filesystem that supports 272*acda97acSIgor Matheus Andrade Torrente`DAX` on a block device that supports `DAX`, they will still be copied into RAM. 273*acda97acSIgor Matheus Andrade Torrente 274*acda97acSIgor Matheus Andrade TorrenteThe DAX code does not work correctly on architectures which have virtually 275*acda97acSIgor Matheus Andrade Torrentemapped caches such as ARM, MIPS and SPARC. 276*acda97acSIgor Matheus Andrade Torrente 277*acda97acSIgor Matheus Andrade TorrenteCalling :c:func:`get_user_pages()` on a range of user memory that has been 278*acda97acSIgor Matheus Andrade Torrentemmaped from a `DAX` file will fail when there are no 'struct page' to describe 279*acda97acSIgor Matheus Andrade Torrentethose pages. This problem has been addressed in some device drivers 280*acda97acSIgor Matheus Andrade Torrenteby adding optional struct page support for pages under the control of 281*acda97acSIgor Matheus Andrade Torrentethe driver (see `CONFIG_NVDIMM_PFN` in ``drivers/nvdimm`` for an example of 282*acda97acSIgor Matheus Andrade Torrentehow to do this). In the non struct page cases `O_DIRECT` reads/writes to 283*acda97acSIgor Matheus Andrade Torrentethose memory ranges from a non-`DAX` file will fail 284*acda97acSIgor Matheus Andrade Torrente 285*acda97acSIgor Matheus Andrade Torrente 286*acda97acSIgor Matheus Andrade Torrente.. note:: 287*acda97acSIgor Matheus Andrade Torrente 288*acda97acSIgor Matheus Andrade Torrente `O_DIRECT` reads/writes _of a `DAX` file do work, it is the memory that 289*acda97acSIgor Matheus Andrade Torrente is being accessed that is key here). Other things that will not work in 290*acda97acSIgor Matheus Andrade Torrente the non struct page case include RDMA, :c:func:`sendfile()` and 291*acda97acSIgor Matheus Andrade Torrente :c:func:`splice()`. 292