1.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later 2 3.. _media-request-api: 4 5Request API 6=========== 7 8The Request API has been designed to allow V4L2 to deal with requirements of 9modern devices (stateless codecs, complex camera pipelines, ...) and APIs 10(Android Codec v2). One such requirement is the ability for devices belonging to 11the same pipeline to reconfigure and collaborate closely on a per-frame basis. 12Another is support of stateless codecs, which require controls to be applied 13to specific frames (aka 'per-frame controls') in order to be used efficiently. 14 15While the initial use-case was V4L2, it can be extended to other subsystems 16as well, as long as they use the media controller. 17 18Supporting these features without the Request API is not always possible and if 19it is, it is terribly inefficient: user-space would have to flush all activity 20on the media pipeline, reconfigure it for the next frame, queue the buffers to 21be processed with that configuration, and wait until they are all available for 22dequeuing before considering the next frame. This defeats the purpose of having 23buffer queues since in practice only one buffer would be queued at a time. 24 25The Request API allows a specific configuration of the pipeline (media 26controller topology + configuration for each media entity) to be associated with 27specific buffers. This allows user-space to schedule several tasks ("requests") 28with different configurations in advance, knowing that the configuration will be 29applied when needed to get the expected result. Configuration values at the time 30of request completion are also available for reading. 31 32General Usage 33------------- 34 35The Request API extends the Media Controller API and cooperates with 36subsystem-specific APIs to support request usage. At the Media Controller 37level, requests are allocated from the supporting Media Controller device 38node. Their life cycle is then managed through the request file descriptors in 39an opaque way. Configuration data, buffer handles and processing results 40stored in requests are accessed through subsystem-specific APIs extended for 41request support, such as V4L2 APIs that take an explicit ``request_fd`` 42parameter. 43 44Request Allocation 45------------------ 46 47User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC` 48for the media device node. This returns a file descriptor representing the 49request. Typically, several such requests will be allocated. 50 51Request Preparation 52------------------- 53 54Standard V4L2 ioctls can then receive a request file descriptor to express the 55fact that the ioctl is part of said request, and is not to be applied 56immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that 57support this. Configurations set with a ``request_fd`` parameter are stored 58instead of being immediately applied, and buffers queued to a request do not 59enter the regular buffer queue until the request itself is queued. 60 61Request Submission 62------------------ 63 64Once the configuration and buffers of the request are specified, it can be 65queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor. 66A request must contain at least one buffer, otherwise ``ENOENT`` is returned. 67A queued request cannot be modified anymore. 68 69.. caution:: 70 For :ref:`memory-to-memory devices <mem2mem>` you can use requests only for 71 output buffers, not for capture buffers. Attempting to add a capture buffer 72 to a request will result in an ``EBADR`` error. 73 74If the request contains configurations for multiple entities, individual drivers 75may synchronize so the requested pipeline's topology is applied before the 76buffers are processed. Media controller drivers do a best effort implementation 77since perfect atomicity may not be possible due to hardware limitations. 78 79.. caution:: 80 81 It is not allowed to mix queuing requests with directly queuing buffers: 82 whichever method is used first locks this in place until 83 :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is 84 :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier 85 a buffer was queued via a request or vice versa will result in an ``EBUSY`` 86 error. 87 88Controls can still be set without a request and are applied immediately, 89regardless of whether a request is in use or not. 90 91.. caution:: 92 93 Setting the same control through a request and also directly can lead to 94 undefined behavior! 95 96User-space can :ref:`poll() <request-func-poll>` a request file descriptor in 97order to wait until the request completes. A request is considered complete 98once all its associated buffers are available for dequeuing and all the 99associated controls have been updated with the values at the time of completion. 100Note that user-space does not need to wait for the request to complete to 101dequeue its buffers: buffers that are available halfway through a request can 102be dequeued independently of the request's state. 103 104A completed request contains the state of the device after the request was 105executed. User-space can query that state by calling 106:ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file 107descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a 108request that has been queued but not yet completed will return ``EBUSY`` 109since the control values might be changed at any time by the driver while the 110request is in flight. 111 112.. _media-request-life-time: 113 114Recycling and Destruction 115------------------------- 116 117Finally, a completed request can either be discarded or be reused. Calling 118:ref:`close() <request-func-close>` on a request file descriptor will make 119that file descriptor unusable and the request will be freed once it is no 120longer in use by the kernel. That is, if the request is queued and then the 121file descriptor is closed, then it won't be freed until the driver completed 122the request. 123 124The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it 125available again. No state is retained by this operation: the request is as 126if it had just been allocated. 127 128Example for a Codec Device 129-------------------------- 130 131For use-cases such as :ref:`codecs <mem2mem>`, the request API can be used 132to associate specific controls to 133be applied by the driver for the OUTPUT buffer, allowing user-space 134to queue many such buffers in advance. It can also take advantage of requests' 135ability to capture the state of controls when the request completes to read back 136information that may be subject to change. 137 138Put into code, after obtaining a request, user-space can assign controls and one 139OUTPUT buffer to it: 140 141.. code-block:: c 142 143 struct v4l2_buffer buf; 144 struct v4l2_ext_controls ctrls; 145 int req_fd; 146 ... 147 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) 148 return errno; 149 ... 150 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 151 ctrls.request_fd = req_fd; 152 if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) 153 return errno; 154 ... 155 buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; 156 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; 157 buf.request_fd = req_fd; 158 if (ioctl(codec_fd, VIDIOC_QBUF, &buf)) 159 return errno; 160 161Note that it is not allowed to use the Request API for CAPTURE buffers 162since there are no per-frame settings to report there. 163 164Once the request is fully prepared, it can be queued to the driver: 165 166.. code-block:: c 167 168 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) 169 return errno; 170 171User-space can then either wait for the request to complete by calling poll() on 172its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will 173want to get CAPTURE buffers as soon as possible and this can be done using a 174regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`: 175 176.. code-block:: c 177 178 struct v4l2_buffer buf; 179 180 memset(&buf, 0, sizeof(buf)); 181 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 182 if (ioctl(codec_fd, VIDIOC_DQBUF, &buf)) 183 return errno; 184 185Note that this example assumes for simplicity that for every OUTPUT buffer 186there will be one CAPTURE buffer, but this does not have to be the case. 187 188We can then, after ensuring that the request is completed via polling the 189request file descriptor, query control values at the time of its completion via 190a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`. 191This is particularly useful for volatile controls for which we want to 192query values as soon as the capture buffer is produced. 193 194.. code-block:: c 195 196 struct pollfd pfd = { .events = POLLPRI, .fd = req_fd }; 197 poll(&pfd, 1, -1); 198 ... 199 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 200 ctrls.request_fd = req_fd; 201 if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls)) 202 return errno; 203 204Once we don't need the request anymore, we can either recycle it for reuse with 205:ref:`MEDIA_REQUEST_IOC_REINIT`... 206 207.. code-block:: c 208 209 if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT)) 210 return errno; 211 212... or close its file descriptor to completely dispose of it. 213 214.. code-block:: c 215 216 close(req_fd); 217 218Example for a Simple Capture Device 219----------------------------------- 220 221With a simple capture device, requests can be used to specify controls to apply 222for a given CAPTURE buffer. 223 224.. code-block:: c 225 226 struct v4l2_buffer buf; 227 struct v4l2_ext_controls ctrls; 228 int req_fd; 229 ... 230 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) 231 return errno; 232 ... 233 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 234 ctrls.request_fd = req_fd; 235 if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) 236 return errno; 237 ... 238 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 239 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; 240 buf.request_fd = req_fd; 241 if (ioctl(camera_fd, VIDIOC_QBUF, &buf)) 242 return errno; 243 244Once the request is fully prepared, it can be queued to the driver: 245 246.. code-block:: c 247 248 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) 249 return errno; 250 251User-space can then dequeue buffers, wait for the request completion, query 252controls and recycle the request as in the M2M example above. 253