Documentation/block/ublk.rst

1 .. SPDX-License-Identifier: GPL-2.0
4 Userspace block device driver (ublk driver)
10 ublk is a generic framework for implementing block device logic from userspace.
11 The motivation behind it is that moving virtual block drivers into userspace,
13 new virtual block device such as ublk-qcow2 (there are several attempts of
16 Userspace block devices are attractive because:
18 - They can be written many programming languages.
19 - They can use libraries that are not available in the kernel.
20 - They can be debugged with tools familiar to application developers.
21 - Crashes do not kernel panic the machine.
22 - Bugs are likely to have a lower security impact than bugs in kernel
24 - They can be installed and updated independently of the kernel.
25 - They can be used to simulate block device easily with user specified
28 ublk block device (``/dev/ublkb*``) is added by ublk driver. Any IO request
33 user block device conveniently, while also generic type block device is
42 ``/dev/ublkb*`` is driven by blk-mq request-based driver. Each request is
48 block driver. It has been observed that using io_uring passthrough command can
49 give better IOPS than block IO; which is why ublk is one of high performance
50 implementation of userspace block device: not only IO request communication is
54 ublk provides control interface to set/get ublk block device parameters.
57 interface. Thus, ublk is generic userspace block device framework.
58 For example, it is easy to setup a ublk device with specified block
64 ublk requires userspace ublk server to handle real block device logic.
66 Below is example of using ``ublksrv`` to provide ublk-based loop device.
68 - add a device::
70      ublk add -t loop -f ublk-loop.img
72 - format with xfs, then use it::
80 - list the devices with their info::
84 - delete the device::
86      ublk del -a
87      ublk del -n $ublk_dev_id
95 -------------
97 ublk driver provides global misc device node (``/dev/ublk-control``) for
100 - ``UBLK_CMD_ADD_DEV``
109 - ``UBLK_CMD_SET_PARAMS`` / ``UBLK_CMD_GET_PARAMS``
116 - ``UBLK_CMD_START_DEV``
118   After the server prepares userspace resources (such as creating per-queue
123 - ``UBLK_CMD_STOP_DEV``
126   ublk server will release resources (such as destroying per-queue pthread &
129 - ``UBLK_CMD_DEL_DEV``
134 - ``UBLK_CMD_GET_QUEUE_AFFINITY``
136   When ``/dev/ublkc`` is added, the driver creates block layer tagset, so
139   set up the per-queue context efficiently, such as bind affine CPUs with IO
142 - ``UBLK_CMD_GET_DEV_INFO``
147 - ``UBLK_CMD_GET_DEV_INFO2``
184 - ``UBLK_CMD_START_USER_RECOVERY``
189   a new process which re-opens ``/dev/ublkc*``. When this command returns, the
192 - ``UBLK_CMD_END_USER_RECOVERY``
200 - user recovery feature description
215   re-issued to the new process after handling ``UBLK_CMD_END_USER_RECOVERY``.
217   double-write since the driver may issue the same I/O request twice. It
218   might be useful to a read-only FS or a VM backend.
226 ublk device becomes container-ware, and device created in one container
230 ----------
232 ublk server needs to create per-queue IO pthread & io_uring for handling IO
233 commands via io_uring passthrough. The per-queue IO pthread
250 - ``UBLK_IO_FETCH_REQ``
256 - ``UBLK_IO_COMMIT_AND_FETCH_REQ``
271 - ``UBLK_IO_NEED_GET_DATA``
274   issued to ublk server without data copy. Then, IO backend of ublk server
277   data copy is done from request pages to this backend's buffer. Finally,
281   ``UBLK_IO_NEED_GET_DATA`` adds one additional round-trip and one
283   should not enable UBLK_F_NEED_GET_DATA. ublk server pre-allocates IO
290 - data copy between ublk server IO buffer and ublk block IO request
292   The driver needs to copy the block IO request pages into the server buffer
297   ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy
303 Zero copy
304 ---------
306 Zero copy is a generic requirement for nbd, fuse or similar drivers. A
310 big requests (IO size >= 256 KB) may benefit a lot from zero copy.
320 .. [#userspace_nbdublk] https://gitlab.com/rwmjones/libnbd/-/tree/nbdublk
324 .. [#stefan] https://lore.kernel.org/linux-block/YoOr6jBfgVm8GvWg@stefanha-x1.localdomain/
326 .. [#xiaoguang] https://lore.kernel.org/linux-block/YoOr6jBfgVm8GvWg@stefanha-x1.localdomain/