1.. Permission is granted to copy, distribute and/or modify this
2.. document under the terms of the GNU Free Documentation License,
3.. Version 1.1 or any later version published by the Free Software
4.. Foundation, with no Invariant Sections, no Front-Cover Texts
5.. and no Back-Cover Texts. A copy of the license is included at
6.. Documentation/userspace-api/media/fdl-appendix.rst.
7..
8.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
9
10.. _mmap:
11
12******************************
13Streaming I/O (Memory Mapping)
14******************************
15
16Input and output devices support this I/O method when the
17``V4L2_CAP_STREAMING`` flag in the ``capabilities`` field of struct
18:c:type:`v4l2_capability` returned by the
19:ref:`VIDIOC_QUERYCAP` ioctl is set. There are two
20streaming methods, to determine if the memory mapping flavor is
21supported applications must call the :ref:`VIDIOC_REQBUFS` ioctl
22with the memory type set to ``V4L2_MEMORY_MMAP``.
23
24Streaming is an I/O method where only pointers to buffers are exchanged
25between application and driver, the data itself is not copied. Memory
26mapping is primarily intended to map buffers in device memory into the
27application's address space. Device memory can be for example the video
28memory on a graphics card with a video capture add-on. However, being
29the most efficient I/O method available for a long time, many other
30drivers support streaming as well, allocating buffers in DMA-able main
31memory.
32
33A driver can support many sets of buffers. Each set is identified by a
34unique buffer type value. The sets are independent and each set can hold
35a different type of data. To access different sets at the same time
36different file descriptors must be used. [#f1]_
37
38To allocate device buffers applications call the
39:ref:`VIDIOC_REQBUFS` ioctl with the desired number
40of buffers and buffer type, for example ``V4L2_BUF_TYPE_VIDEO_CAPTURE``.
41This ioctl can also be used to change the number of buffers or to free
42the allocated memory, provided none of the buffers are still mapped.
43
44Before applications can access the buffers they must map them into their
45address space with the :ref:`mmap() <func-mmap>` function. The
46location of the buffers in device memory can be determined with the
47:ref:`VIDIOC_QUERYBUF` ioctl. In the single-planar
48API case, the ``m.offset`` and ``length`` returned in a struct
49:c:type:`v4l2_buffer` are passed as sixth and second
50parameter to the :ref:`mmap() <func-mmap>` function. When using the
51multi-planar API, struct :c:type:`v4l2_buffer` contains an
52array of struct :c:type:`v4l2_plane` structures, each
53containing its own ``m.offset`` and ``length``. When using the
54multi-planar API, every plane of every buffer has to be mapped
55separately, so the number of calls to :ref:`mmap() <func-mmap>` should
56be equal to number of buffers times number of planes in each buffer. The
57offset and length values must not be modified. Remember, the buffers are
58allocated in physical memory, as opposed to virtual memory, which can be
59swapped out to disk. Applications should free the buffers as soon as
60possible with the :ref:`munmap() <func-munmap>` function.
61
62Example: Mapping buffers in the single-planar API
63=================================================
64
65.. code-block:: c
66
67    struct v4l2_requestbuffers reqbuf;
68    struct {
69	void *start;
70	size_t length;
71    } *buffers;
72    unsigned int i;
73
74    memset(&reqbuf, 0, sizeof(reqbuf));
75    reqbuf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
76    reqbuf.memory = V4L2_MEMORY_MMAP;
77    reqbuf.count = 20;
78
79    if (-1 == ioctl (fd, VIDIOC_REQBUFS, &reqbuf)) {
80	if (errno == EINVAL)
81	    printf("Video capturing or mmap-streaming is not supported\\n");
82	else
83	    perror("VIDIOC_REQBUFS");
84
85	exit(EXIT_FAILURE);
86    }
87
88    /* We want at least five buffers. */
89
90    if (reqbuf.count < 5) {
91	/* You may need to free the buffers here. */
92	printf("Not enough buffer memory\\n");
93	exit(EXIT_FAILURE);
94    }
95
96    buffers = calloc(reqbuf.count, sizeof(*buffers));
97    assert(buffers != NULL);
98
99    for (i = 0; i < reqbuf.count; i++) {
100	struct v4l2_buffer buffer;
101
102	memset(&buffer, 0, sizeof(buffer));
103	buffer.type = reqbuf.type;
104	buffer.memory = V4L2_MEMORY_MMAP;
105	buffer.index = i;
106
107	if (-1 == ioctl (fd, VIDIOC_QUERYBUF, &buffer)) {
108	    perror("VIDIOC_QUERYBUF");
109	    exit(EXIT_FAILURE);
110	}
111
112	buffers[i].length = buffer.length; /* remember for munmap() */
113
114	buffers[i].start = mmap(NULL, buffer.length,
115		    PROT_READ | PROT_WRITE, /* recommended */
116		    MAP_SHARED,             /* recommended */
117		    fd, buffer.m.offset);
118
119	if (MAP_FAILED == buffers[i].start) {
120	    /* If you do not exit here you should unmap() and free()
121	       the buffers mapped so far. */
122	    perror("mmap");
123	    exit(EXIT_FAILURE);
124	}
125    }
126
127    /* Cleanup. */
128
129    for (i = 0; i < reqbuf.count; i++)
130	munmap(buffers[i].start, buffers[i].length);
131
132
133Example: Mapping buffers in the multi-planar API
134================================================
135
136.. code-block:: c
137
138    struct v4l2_requestbuffers reqbuf;
139    /* Our current format uses 3 planes per buffer */
140    #define FMT_NUM_PLANES = 3
141
142    struct {
143	void *start[FMT_NUM_PLANES];
144	size_t length[FMT_NUM_PLANES];
145    } *buffers;
146    unsigned int i, j;
147
148    memset(&reqbuf, 0, sizeof(reqbuf));
149    reqbuf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
150    reqbuf.memory = V4L2_MEMORY_MMAP;
151    reqbuf.count = 20;
152
153    if (ioctl(fd, VIDIOC_REQBUFS, &reqbuf) < 0) {
154	if (errno == EINVAL)
155	    printf("Video capturing or mmap-streaming is not supported\\n");
156	else
157	    perror("VIDIOC_REQBUFS");
158
159	exit(EXIT_FAILURE);
160    }
161
162    /* We want at least five buffers. */
163
164    if (reqbuf.count < 5) {
165	/* You may need to free the buffers here. */
166	printf("Not enough buffer memory\\n");
167	exit(EXIT_FAILURE);
168    }
169
170    buffers = calloc(reqbuf.count, sizeof(*buffers));
171    assert(buffers != NULL);
172
173    for (i = 0; i < reqbuf.count; i++) {
174	struct v4l2_buffer buffer;
175	struct v4l2_plane planes[FMT_NUM_PLANES];
176
177	memset(&buffer, 0, sizeof(buffer));
178	buffer.type = reqbuf.type;
179	buffer.memory = V4L2_MEMORY_MMAP;
180	buffer.index = i;
181	/* length in struct v4l2_buffer in multi-planar API stores the size
182	 * of planes array. */
183	buffer.length = FMT_NUM_PLANES;
184	buffer.m.planes = planes;
185
186	if (ioctl(fd, VIDIOC_QUERYBUF, &buffer) < 0) {
187	    perror("VIDIOC_QUERYBUF");
188	    exit(EXIT_FAILURE);
189	}
190
191	/* Every plane has to be mapped separately */
192	for (j = 0; j < FMT_NUM_PLANES; j++) {
193	    buffers[i].length[j] = buffer.m.planes[j].length; /* remember for munmap() */
194
195	    buffers[i].start[j] = mmap(NULL, buffer.m.planes[j].length,
196		     PROT_READ | PROT_WRITE, /* recommended */
197		     MAP_SHARED,             /* recommended */
198		     fd, buffer.m.planes[j].m.offset);
199
200	    if (MAP_FAILED == buffers[i].start[j]) {
201		/* If you do not exit here you should unmap() and free()
202		   the buffers and planes mapped so far. */
203		perror("mmap");
204		exit(EXIT_FAILURE);
205	    }
206	}
207    }
208
209    /* Cleanup. */
210
211    for (i = 0; i < reqbuf.count; i++)
212	for (j = 0; j < FMT_NUM_PLANES; j++)
213	    munmap(buffers[i].start[j], buffers[i].length[j]);
214
215Conceptually streaming drivers maintain two buffer queues, an incoming
216and an outgoing queue. They separate the synchronous capture or output
217operation locked to a video clock from the application which is subject
218to random disk or network delays and preemption by other processes,
219thereby reducing the probability of data loss. The queues are organized
220as FIFOs, buffers will be output in the order enqueued in the incoming
221FIFO, and were captured in the order dequeued from the outgoing FIFO.
222
223The driver may require a minimum number of buffers enqueued at all times
224to function, apart of this no limit exists on the number of buffers
225applications can enqueue in advance, or dequeue and process. They can
226also enqueue in a different order than buffers have been dequeued, and
227the driver can *fill* enqueued *empty* buffers in any order.  [#f2]_ The
228index number of a buffer (struct :c:type:`v4l2_buffer`
229``index``) plays no role here, it only identifies the buffer.
230
231Initially all mapped buffers are in dequeued state, inaccessible by the
232driver. For capturing applications it is customary to first enqueue all
233mapped buffers, then to start capturing and enter the read loop. Here
234the application waits until a filled buffer can be dequeued, and
235re-enqueues the buffer when the data is no longer needed. Output
236applications fill and enqueue buffers, when enough buffers are stacked
237up the output is started with :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>`.
238In the write loop, when the application runs out of free buffers, it
239must wait until an empty buffer can be dequeued and reused.
240
241To enqueue and dequeue a buffer applications use the
242:ref:`VIVIOC_QBUF <VIDIOC_QBUF>` and :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`
243ioctl. The status of a buffer being mapped, enqueued, full or empty can
244be determined at any time using the :ref:`VIDIOC_QUERYBUF` ioctl. Two
245methods exist to suspend execution of the application until one or more
246buffers can be dequeued.  By default :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`
247blocks when no buffer is in the outgoing queue. When the ``O_NONBLOCK``
248flag was given to the :ref:`open() <func-open>` function,
249:ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` returns immediately with an ``EAGAIN``
250error code when no buffer is available. The :ref:`select() <func-select>`
251or :ref:`poll() <func-poll>` functions are always available.
252
253To start and stop capturing or output applications call the
254:ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>` and :ref:`VIDIOC_STREAMOFF
255<VIDIOC_STREAMON>` ioctl.
256
257.. note:::ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>`
258   removes all buffers from both queues as a side effect. Since there is
259   no notion of doing anything "now" on a multitasking system, if an
260   application needs to synchronize with another event it should examine
261   the struct ::c:type:`v4l2_buffer` ``timestamp`` of captured
262   or outputted buffers.
263
264Drivers implementing memory mapping I/O must support the
265:ref:`VIDIOC_REQBUFS <VIDIOC_REQBUFS>`, :ref:`VIDIOC_QUERYBUF
266<VIDIOC_QUERYBUF>`, :ref:`VIDIOC_QBUF <VIDIOC_QBUF>`, :ref:`VIDIOC_DQBUF
267<VIDIOC_QBUF>`, :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>`
268and :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` ioctls, the :ref:`mmap()
269<func-mmap>`, :ref:`munmap() <func-munmap>`, :ref:`select()
270<func-select>` and :ref:`poll() <func-poll>` function. [#f3]_
271
272[capture example]
273
274.. [#f1]
275   One could use one file descriptor and set the buffer type field
276   accordingly when calling :ref:`VIDIOC_QBUF` etc.,
277   but it makes the :ref:`select() <func-select>` function ambiguous. We also
278   like the clean approach of one file descriptor per logical stream.
279   Video overlay for example is also a logical stream, although the CPU
280   is not needed for continuous operation.
281
282.. [#f2]
283   Random enqueue order permits applications processing images out of
284   order (such as video codecs) to return buffers earlier, reducing the
285   probability of data loss. Random fill order allows drivers to reuse
286   buffers on a LIFO-basis, taking advantage of caches holding
287   scatter-gather lists and the like.
288
289.. [#f3]
290   At the driver level :ref:`select() <func-select>` and :ref:`poll() <func-poll>` are
291   the same, and :ref:`select() <func-select>` is too important to be optional.
292   The rest should be evident.
293