154f38fcaSMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0 254f38fcaSMauro Carvalho Chehab 354f38fcaSMauro Carvalho Chehab.. _stateless_decoder: 454f38fcaSMauro Carvalho Chehab 554f38fcaSMauro Carvalho Chehab************************************************** 654f38fcaSMauro Carvalho ChehabMemory-to-memory Stateless Video Decoder Interface 754f38fcaSMauro Carvalho Chehab************************************************** 854f38fcaSMauro Carvalho Chehab 954f38fcaSMauro Carvalho ChehabA stateless decoder is a decoder that works without retaining any kind of state 1054f38fcaSMauro Carvalho Chehabbetween processed frames. This means that each frame is decoded independently 1154f38fcaSMauro Carvalho Chehabof any previous and future frames, and that the client is responsible for 1254f38fcaSMauro Carvalho Chehabmaintaining the decoding state and providing it to the decoder with each 1354f38fcaSMauro Carvalho Chehabdecoding request. This is in contrast to the stateful video decoder interface, 1454f38fcaSMauro Carvalho Chehabwhere the hardware and driver maintain the decoding state and all the client 1554f38fcaSMauro Carvalho Chehabhas to do is to provide the raw encoded stream and dequeue decoded frames in 1654f38fcaSMauro Carvalho Chehabdisplay order. 1754f38fcaSMauro Carvalho Chehab 1854f38fcaSMauro Carvalho ChehabThis section describes how user-space ("the client") is expected to communicate 1954f38fcaSMauro Carvalho Chehabwith stateless decoders in order to successfully decode an encoded stream. 2054f38fcaSMauro Carvalho ChehabCompared to stateful codecs, the decoder/client sequence is simpler, but the 2154f38fcaSMauro Carvalho Chehabcost of this simplicity is extra complexity in the client which is responsible 2254f38fcaSMauro Carvalho Chehabfor maintaining a consistent decoding state. 2354f38fcaSMauro Carvalho Chehab 2454f38fcaSMauro Carvalho ChehabStateless decoders make use of the :ref:`media-request-api`. A stateless 2554f38fcaSMauro Carvalho Chehabdecoder must expose the ``V4L2_BUF_CAP_SUPPORTS_REQUESTS`` capability on its 2654f38fcaSMauro Carvalho Chehab``OUTPUT`` queue when :c:func:`VIDIOC_REQBUFS` or :c:func:`VIDIOC_CREATE_BUFS` 2754f38fcaSMauro Carvalho Chehabare invoked. 2854f38fcaSMauro Carvalho Chehab 2954f38fcaSMauro Carvalho ChehabDepending on the encoded formats supported by the decoder, a single decoded 3054f38fcaSMauro Carvalho Chehabframe may be the result of several decode requests (for instance, H.264 streams 3154f38fcaSMauro Carvalho Chehabwith multiple slices per frame). Decoders that support such formats must also 3254f38fcaSMauro Carvalho Chehabexpose the ``V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF`` capability on their 3354f38fcaSMauro Carvalho Chehab``OUTPUT`` queue. 3454f38fcaSMauro Carvalho Chehab 3554f38fcaSMauro Carvalho ChehabQuerying capabilities 3654f38fcaSMauro Carvalho Chehab===================== 3754f38fcaSMauro Carvalho Chehab 3854f38fcaSMauro Carvalho Chehab1. To enumerate the set of coded formats supported by the decoder, the client 3954f38fcaSMauro Carvalho Chehab calls :c:func:`VIDIOC_ENUM_FMT` on the ``OUTPUT`` queue. 4054f38fcaSMauro Carvalho Chehab 4154f38fcaSMauro Carvalho Chehab * The driver must always return the full set of supported ``OUTPUT`` formats, 4254f38fcaSMauro Carvalho Chehab irrespective of the format currently set on the ``CAPTURE`` queue. 4354f38fcaSMauro Carvalho Chehab 4454f38fcaSMauro Carvalho Chehab * Simultaneously, the driver must restrain the set of values returned by 4554f38fcaSMauro Carvalho Chehab codec-specific capability controls (such as H.264 profiles) to the set 4654f38fcaSMauro Carvalho Chehab actually supported by the hardware. 4754f38fcaSMauro Carvalho Chehab 4854f38fcaSMauro Carvalho Chehab2. To enumerate the set of supported raw formats, the client calls 4954f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_ENUM_FMT` on the ``CAPTURE`` queue. 5054f38fcaSMauro Carvalho Chehab 5154f38fcaSMauro Carvalho Chehab * The driver must return only the formats supported for the format currently 5254f38fcaSMauro Carvalho Chehab active on the ``OUTPUT`` queue. 5354f38fcaSMauro Carvalho Chehab 5454f38fcaSMauro Carvalho Chehab * Depending on the currently set ``OUTPUT`` format, the set of supported raw 5554f38fcaSMauro Carvalho Chehab formats may depend on the value of some codec-dependent controls. 5654f38fcaSMauro Carvalho Chehab The client is responsible for making sure that these controls are set 5754f38fcaSMauro Carvalho Chehab before querying the ``CAPTURE`` queue. Failure to do so will result in the 5854f38fcaSMauro Carvalho Chehab default values for these controls being used, and a returned set of formats 5954f38fcaSMauro Carvalho Chehab that may not be usable for the media the client is trying to decode. 6054f38fcaSMauro Carvalho Chehab 6154f38fcaSMauro Carvalho Chehab3. The client may use :c:func:`VIDIOC_ENUM_FRAMESIZES` to detect supported 6254f38fcaSMauro Carvalho Chehab resolutions for a given format, passing desired pixel format in 6354f38fcaSMauro Carvalho Chehab :c:type:`v4l2_frmsizeenum`'s ``pixel_format``. 6454f38fcaSMauro Carvalho Chehab 6554f38fcaSMauro Carvalho Chehab4. Supported profiles and levels for the current ``OUTPUT`` format, if 6654f38fcaSMauro Carvalho Chehab applicable, may be queried using their respective controls via 6754f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_QUERYCTRL`. 6854f38fcaSMauro Carvalho Chehab 6954f38fcaSMauro Carvalho ChehabInitialization 7054f38fcaSMauro Carvalho Chehab============== 7154f38fcaSMauro Carvalho Chehab 7254f38fcaSMauro Carvalho Chehab1. Set the coded format on the ``OUTPUT`` queue via :c:func:`VIDIOC_S_FMT`. 7354f38fcaSMauro Carvalho Chehab 7454f38fcaSMauro Carvalho Chehab * **Required fields:** 7554f38fcaSMauro Carvalho Chehab 7654f38fcaSMauro Carvalho Chehab ``type`` 7754f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 7854f38fcaSMauro Carvalho Chehab 7954f38fcaSMauro Carvalho Chehab ``pixelformat`` 8054f38fcaSMauro Carvalho Chehab a coded pixel format. 8154f38fcaSMauro Carvalho Chehab 8254f38fcaSMauro Carvalho Chehab ``width``, ``height`` 8354f38fcaSMauro Carvalho Chehab coded width and height parsed from the stream. 8454f38fcaSMauro Carvalho Chehab 8554f38fcaSMauro Carvalho Chehab other fields 8654f38fcaSMauro Carvalho Chehab follow standard semantics. 8754f38fcaSMauro Carvalho Chehab 8854f38fcaSMauro Carvalho Chehab .. note:: 8954f38fcaSMauro Carvalho Chehab 9054f38fcaSMauro Carvalho Chehab Changing the ``OUTPUT`` format may change the currently set ``CAPTURE`` 9154f38fcaSMauro Carvalho Chehab format. The driver will derive a new ``CAPTURE`` format from the 9254f38fcaSMauro Carvalho Chehab ``OUTPUT`` format being set, including resolution, colorimetry 9354f38fcaSMauro Carvalho Chehab parameters, etc. If the client needs a specific ``CAPTURE`` format, 9454f38fcaSMauro Carvalho Chehab it must adjust it afterwards. 9554f38fcaSMauro Carvalho Chehab 9654f38fcaSMauro Carvalho Chehab2. Call :c:func:`VIDIOC_S_EXT_CTRLS` to set all the controls (parsed headers, 9754f38fcaSMauro Carvalho Chehab etc.) required by the ``OUTPUT`` format to enumerate the ``CAPTURE`` formats. 9854f38fcaSMauro Carvalho Chehab 9954f38fcaSMauro Carvalho Chehab3. Call :c:func:`VIDIOC_G_FMT` for ``CAPTURE`` queue to get the format for the 10054f38fcaSMauro Carvalho Chehab destination buffers parsed/decoded from the bytestream. 10154f38fcaSMauro Carvalho Chehab 10254f38fcaSMauro Carvalho Chehab * **Required fields:** 10354f38fcaSMauro Carvalho Chehab 10454f38fcaSMauro Carvalho Chehab ``type`` 10554f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 10654f38fcaSMauro Carvalho Chehab 10754f38fcaSMauro Carvalho Chehab * **Returned fields:** 10854f38fcaSMauro Carvalho Chehab 10954f38fcaSMauro Carvalho Chehab ``width``, ``height`` 11054f38fcaSMauro Carvalho Chehab frame buffer resolution for the decoded frames. 11154f38fcaSMauro Carvalho Chehab 11254f38fcaSMauro Carvalho Chehab ``pixelformat`` 11354f38fcaSMauro Carvalho Chehab pixel format for decoded frames. 11454f38fcaSMauro Carvalho Chehab 11554f38fcaSMauro Carvalho Chehab ``num_planes`` (for _MPLANE ``type`` only) 11654f38fcaSMauro Carvalho Chehab number of planes for pixelformat. 11754f38fcaSMauro Carvalho Chehab 11854f38fcaSMauro Carvalho Chehab ``sizeimage``, ``bytesperline`` 11954f38fcaSMauro Carvalho Chehab as per standard semantics; matching frame buffer format. 12054f38fcaSMauro Carvalho Chehab 12154f38fcaSMauro Carvalho Chehab .. note:: 12254f38fcaSMauro Carvalho Chehab 12354f38fcaSMauro Carvalho Chehab The value of ``pixelformat`` may be any pixel format supported for the 12454f38fcaSMauro Carvalho Chehab ``OUTPUT`` format, based on the hardware capabilities. It is suggested 12554f38fcaSMauro Carvalho Chehab that the driver chooses the preferred/optimal format for the current 12654f38fcaSMauro Carvalho Chehab configuration. For example, a YUV format may be preferred over an RGB 12754f38fcaSMauro Carvalho Chehab format, if an additional conversion step would be required for RGB. 12854f38fcaSMauro Carvalho Chehab 12954f38fcaSMauro Carvalho Chehab4. *[optional]* Enumerate ``CAPTURE`` formats via :c:func:`VIDIOC_ENUM_FMT` on 13054f38fcaSMauro Carvalho Chehab the ``CAPTURE`` queue. The client may use this ioctl to discover which 13154f38fcaSMauro Carvalho Chehab alternative raw formats are supported for the current ``OUTPUT`` format and 13254f38fcaSMauro Carvalho Chehab select one of them via :c:func:`VIDIOC_S_FMT`. 13354f38fcaSMauro Carvalho Chehab 13454f38fcaSMauro Carvalho Chehab .. note:: 13554f38fcaSMauro Carvalho Chehab 13654f38fcaSMauro Carvalho Chehab The driver will return only formats supported for the currently selected 13754f38fcaSMauro Carvalho Chehab ``OUTPUT`` format and currently set controls, even if more formats may be 13854f38fcaSMauro Carvalho Chehab supported by the decoder in general. 13954f38fcaSMauro Carvalho Chehab 14054f38fcaSMauro Carvalho Chehab For example, a decoder may support YUV and RGB formats for 14154f38fcaSMauro Carvalho Chehab resolutions 1920x1088 and lower, but only YUV for higher resolutions (due 14254f38fcaSMauro Carvalho Chehab to hardware limitations). After setting a resolution of 1920x1088 or lower 14354f38fcaSMauro Carvalho Chehab as the ``OUTPUT`` format, :c:func:`VIDIOC_ENUM_FMT` may return a set of 14454f38fcaSMauro Carvalho Chehab YUV and RGB pixel formats, but after setting a resolution higher than 14554f38fcaSMauro Carvalho Chehab 1920x1088, the driver will not return RGB pixel formats, since they are 14654f38fcaSMauro Carvalho Chehab unsupported for this resolution. 14754f38fcaSMauro Carvalho Chehab 14854f38fcaSMauro Carvalho Chehab5. *[optional]* Choose a different ``CAPTURE`` format than suggested via 14954f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_S_FMT` on ``CAPTURE`` queue. It is possible for the client to 15054f38fcaSMauro Carvalho Chehab choose a different format than selected/suggested by the driver in 15154f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_G_FMT`. 15254f38fcaSMauro Carvalho Chehab 15354f38fcaSMauro Carvalho Chehab * **Required fields:** 15454f38fcaSMauro Carvalho Chehab 15554f38fcaSMauro Carvalho Chehab ``type`` 15654f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 15754f38fcaSMauro Carvalho Chehab 15854f38fcaSMauro Carvalho Chehab ``pixelformat`` 15954f38fcaSMauro Carvalho Chehab a raw pixel format. 16054f38fcaSMauro Carvalho Chehab 16154f38fcaSMauro Carvalho Chehab ``width``, ``height`` 16254f38fcaSMauro Carvalho Chehab frame buffer resolution of the decoded stream; typically unchanged from 16354f38fcaSMauro Carvalho Chehab what was returned with :c:func:`VIDIOC_G_FMT`, but it may be different 16454f38fcaSMauro Carvalho Chehab if the hardware supports composition and/or scaling. 16554f38fcaSMauro Carvalho Chehab 16654f38fcaSMauro Carvalho Chehab After performing this step, the client must perform step 3 again in order 16754f38fcaSMauro Carvalho Chehab to obtain up-to-date information about the buffers size and layout. 16854f38fcaSMauro Carvalho Chehab 16954f38fcaSMauro Carvalho Chehab6. Allocate source (bytestream) buffers via :c:func:`VIDIOC_REQBUFS` on 17054f38fcaSMauro Carvalho Chehab ``OUTPUT`` queue. 17154f38fcaSMauro Carvalho Chehab 17254f38fcaSMauro Carvalho Chehab * **Required fields:** 17354f38fcaSMauro Carvalho Chehab 17454f38fcaSMauro Carvalho Chehab ``count`` 17554f38fcaSMauro Carvalho Chehab requested number of buffers to allocate; greater than zero. 17654f38fcaSMauro Carvalho Chehab 17754f38fcaSMauro Carvalho Chehab ``type`` 17854f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 17954f38fcaSMauro Carvalho Chehab 18054f38fcaSMauro Carvalho Chehab ``memory`` 18154f38fcaSMauro Carvalho Chehab follows standard semantics. 18254f38fcaSMauro Carvalho Chehab 183*1073f441SPaul Kocialkowski * **Returned fields:** 18454f38fcaSMauro Carvalho Chehab 18554f38fcaSMauro Carvalho Chehab ``count`` 18654f38fcaSMauro Carvalho Chehab actual number of buffers allocated. 18754f38fcaSMauro Carvalho Chehab 18854f38fcaSMauro Carvalho Chehab * If required, the driver will adjust ``count`` to be equal or bigger to the 18954f38fcaSMauro Carvalho Chehab minimum of required number of ``OUTPUT`` buffers for the given format and 19054f38fcaSMauro Carvalho Chehab requested count. The client must check this value after the ioctl returns 19154f38fcaSMauro Carvalho Chehab to get the actual number of buffers allocated. 19254f38fcaSMauro Carvalho Chehab 19354f38fcaSMauro Carvalho Chehab7. Allocate destination (raw format) buffers via :c:func:`VIDIOC_REQBUFS` on the 19454f38fcaSMauro Carvalho Chehab ``CAPTURE`` queue. 19554f38fcaSMauro Carvalho Chehab 19654f38fcaSMauro Carvalho Chehab * **Required fields:** 19754f38fcaSMauro Carvalho Chehab 19854f38fcaSMauro Carvalho Chehab ``count`` 19954f38fcaSMauro Carvalho Chehab requested number of buffers to allocate; greater than zero. The client 20054f38fcaSMauro Carvalho Chehab is responsible for deducing the minimum number of buffers required 20154f38fcaSMauro Carvalho Chehab for the stream to be properly decoded (taking e.g. reference frames 20254f38fcaSMauro Carvalho Chehab into account) and pass an equal or bigger number. 20354f38fcaSMauro Carvalho Chehab 20454f38fcaSMauro Carvalho Chehab ``type`` 20554f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 20654f38fcaSMauro Carvalho Chehab 20754f38fcaSMauro Carvalho Chehab ``memory`` 20854f38fcaSMauro Carvalho Chehab follows standard semantics. ``V4L2_MEMORY_USERPTR`` is not supported 20954f38fcaSMauro Carvalho Chehab for ``CAPTURE`` buffers. 21054f38fcaSMauro Carvalho Chehab 211*1073f441SPaul Kocialkowski * **Returned fields:** 21254f38fcaSMauro Carvalho Chehab 21354f38fcaSMauro Carvalho Chehab ``count`` 21454f38fcaSMauro Carvalho Chehab adjusted to allocated number of buffers, in case the codec requires 21554f38fcaSMauro Carvalho Chehab more buffers than requested. 21654f38fcaSMauro Carvalho Chehab 21754f38fcaSMauro Carvalho Chehab * The driver must adjust count to the minimum of required number of 21854f38fcaSMauro Carvalho Chehab ``CAPTURE`` buffers for the current format, stream configuration and 21954f38fcaSMauro Carvalho Chehab requested count. The client must check this value after the ioctl 22054f38fcaSMauro Carvalho Chehab returns to get the number of buffers allocated. 22154f38fcaSMauro Carvalho Chehab 22254f38fcaSMauro Carvalho Chehab8. Allocate requests (likely one per ``OUTPUT`` buffer) via 22354f38fcaSMauro Carvalho Chehab :c:func:`MEDIA_IOC_REQUEST_ALLOC` on the media device. 22454f38fcaSMauro Carvalho Chehab 22554f38fcaSMauro Carvalho Chehab9. Start streaming on both ``OUTPUT`` and ``CAPTURE`` queues via 22654f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_STREAMON`. 22754f38fcaSMauro Carvalho Chehab 22854f38fcaSMauro Carvalho ChehabDecoding 22954f38fcaSMauro Carvalho Chehab======== 23054f38fcaSMauro Carvalho Chehab 23154f38fcaSMauro Carvalho ChehabFor each frame, the client is responsible for submitting at least one request to 23254f38fcaSMauro Carvalho Chehabwhich the following is attached: 23354f38fcaSMauro Carvalho Chehab 23454f38fcaSMauro Carvalho Chehab* The amount of encoded data expected by the codec for its current 23554f38fcaSMauro Carvalho Chehab configuration, as a buffer submitted to the ``OUTPUT`` queue. Typically, this 23654f38fcaSMauro Carvalho Chehab corresponds to one frame worth of encoded data, but some formats may allow (or 23754f38fcaSMauro Carvalho Chehab require) different amounts per unit. 23854f38fcaSMauro Carvalho Chehab* All the metadata needed to decode the submitted encoded data, in the form of 23954f38fcaSMauro Carvalho Chehab controls relevant to the format being decoded. 24054f38fcaSMauro Carvalho Chehab 24154f38fcaSMauro Carvalho ChehabThe amount of data and contents of the source ``OUTPUT`` buffer, as well as the 24254f38fcaSMauro Carvalho Chehabcontrols that must be set on the request, depend on the active coded pixel 24354f38fcaSMauro Carvalho Chehabformat and might be affected by codec-specific extended controls, as stated in 24454f38fcaSMauro Carvalho Chehabdocumentation of each format. 24554f38fcaSMauro Carvalho Chehab 24654f38fcaSMauro Carvalho ChehabIf there is a possibility that the decoded frame will require one or more 24754f38fcaSMauro Carvalho Chehabdecode requests after the current one in order to be produced, then the client 24854f38fcaSMauro Carvalho Chehabmust set the ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag on the ``OUTPUT`` 24954f38fcaSMauro Carvalho Chehabbuffer. This will result in the (potentially partially) decoded ``CAPTURE`` 25054f38fcaSMauro Carvalho Chehabbuffer not being made available for dequeueing, and reused for the next decode 25154f38fcaSMauro Carvalho Chehabrequest if the timestamp of the next ``OUTPUT`` buffer has not changed. 25254f38fcaSMauro Carvalho Chehab 25354f38fcaSMauro Carvalho ChehabA typical frame would thus be decoded using the following sequence: 25454f38fcaSMauro Carvalho Chehab 25554f38fcaSMauro Carvalho Chehab1. Queue an ``OUTPUT`` buffer containing one unit of encoded bytestream data for 25654f38fcaSMauro Carvalho Chehab the decoding request, using :c:func:`VIDIOC_QBUF`. 25754f38fcaSMauro Carvalho Chehab 25854f38fcaSMauro Carvalho Chehab * **Required fields:** 25954f38fcaSMauro Carvalho Chehab 26054f38fcaSMauro Carvalho Chehab ``index`` 26154f38fcaSMauro Carvalho Chehab index of the buffer being queued. 26254f38fcaSMauro Carvalho Chehab 26354f38fcaSMauro Carvalho Chehab ``type`` 26454f38fcaSMauro Carvalho Chehab type of the buffer. 26554f38fcaSMauro Carvalho Chehab 26654f38fcaSMauro Carvalho Chehab ``bytesused`` 26754f38fcaSMauro Carvalho Chehab number of bytes taken by the encoded data frame in the buffer. 26854f38fcaSMauro Carvalho Chehab 26954f38fcaSMauro Carvalho Chehab ``flags`` 27054f38fcaSMauro Carvalho Chehab the ``V4L2_BUF_FLAG_REQUEST_FD`` flag must be set. Additionally, if 27154f38fcaSMauro Carvalho Chehab we are not sure that the current decode request is the last one needed 27254f38fcaSMauro Carvalho Chehab to produce a fully decoded frame, then 27354f38fcaSMauro Carvalho Chehab ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` must also be set. 27454f38fcaSMauro Carvalho Chehab 27554f38fcaSMauro Carvalho Chehab ``request_fd`` 27654f38fcaSMauro Carvalho Chehab must be set to the file descriptor of the decoding request. 27754f38fcaSMauro Carvalho Chehab 27854f38fcaSMauro Carvalho Chehab ``timestamp`` 27954f38fcaSMauro Carvalho Chehab must be set to a unique value per frame. This value will be propagated 28054f38fcaSMauro Carvalho Chehab into the decoded frame's buffer and can also be used to use this frame 28154f38fcaSMauro Carvalho Chehab as the reference of another. If using multiple decode requests per 28254f38fcaSMauro Carvalho Chehab frame, then the timestamps of all the ``OUTPUT`` buffers for a given 28354f38fcaSMauro Carvalho Chehab frame must be identical. If the timestamp changes, then the currently 28454f38fcaSMauro Carvalho Chehab held ``CAPTURE`` buffer will be made available for dequeuing and the 28554f38fcaSMauro Carvalho Chehab current request will work on a new ``CAPTURE`` buffer. 28654f38fcaSMauro Carvalho Chehab 28754f38fcaSMauro Carvalho Chehab2. Set the codec-specific controls for the decoding request, using 28854f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_S_EXT_CTRLS`. 28954f38fcaSMauro Carvalho Chehab 29054f38fcaSMauro Carvalho Chehab * **Required fields:** 29154f38fcaSMauro Carvalho Chehab 29254f38fcaSMauro Carvalho Chehab ``which`` 29354f38fcaSMauro Carvalho Chehab must be ``V4L2_CTRL_WHICH_REQUEST_VAL``. 29454f38fcaSMauro Carvalho Chehab 29554f38fcaSMauro Carvalho Chehab ``request_fd`` 29654f38fcaSMauro Carvalho Chehab must be set to the file descriptor of the decoding request. 29754f38fcaSMauro Carvalho Chehab 29854f38fcaSMauro Carvalho Chehab other fields 29954f38fcaSMauro Carvalho Chehab other fields are set as usual when setting controls. The ``controls`` 30054f38fcaSMauro Carvalho Chehab array must contain all the codec-specific controls required to decode 30154f38fcaSMauro Carvalho Chehab a frame. 30254f38fcaSMauro Carvalho Chehab 30354f38fcaSMauro Carvalho Chehab .. note:: 30454f38fcaSMauro Carvalho Chehab 30554f38fcaSMauro Carvalho Chehab It is possible to specify the controls in different invocations of 30654f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_S_EXT_CTRLS`, or to overwrite a previously set control, as 30754f38fcaSMauro Carvalho Chehab long as ``request_fd`` and ``which`` are properly set. The controls state 30854f38fcaSMauro Carvalho Chehab at the moment of request submission is the one that will be considered. 30954f38fcaSMauro Carvalho Chehab 31054f38fcaSMauro Carvalho Chehab .. note:: 31154f38fcaSMauro Carvalho Chehab 31254f38fcaSMauro Carvalho Chehab The order in which steps 1 and 2 take place is interchangeable. 31354f38fcaSMauro Carvalho Chehab 31454f38fcaSMauro Carvalho Chehab3. Submit the request by invoking :c:func:`MEDIA_REQUEST_IOC_QUEUE` on the 31554f38fcaSMauro Carvalho Chehab request FD. 31654f38fcaSMauro Carvalho Chehab 31754f38fcaSMauro Carvalho Chehab If the request is submitted without an ``OUTPUT`` buffer, or if some of the 31854f38fcaSMauro Carvalho Chehab required controls are missing from the request, then 31954f38fcaSMauro Carvalho Chehab :c:func:`MEDIA_REQUEST_IOC_QUEUE` will return ``-ENOENT``. If more than one 32054f38fcaSMauro Carvalho Chehab ``OUTPUT`` buffer is queued, then it will return ``-EINVAL``. 32154f38fcaSMauro Carvalho Chehab :c:func:`MEDIA_REQUEST_IOC_QUEUE` returning non-zero means that no 32254f38fcaSMauro Carvalho Chehab ``CAPTURE`` buffer will be produced for this request. 32354f38fcaSMauro Carvalho Chehab 32454f38fcaSMauro Carvalho Chehab``CAPTURE`` buffers must not be part of the request, and are queued 32554f38fcaSMauro Carvalho Chehabindependently. They are returned in decode order (i.e. the same order as coded 32654f38fcaSMauro Carvalho Chehabframes were submitted to the ``OUTPUT`` queue). 32754f38fcaSMauro Carvalho Chehab 32854f38fcaSMauro Carvalho ChehabRuntime decoding errors are signaled by the dequeued ``CAPTURE`` buffers 32954f38fcaSMauro Carvalho Chehabcarrying the ``V4L2_BUF_FLAG_ERROR`` flag. If a decoded reference frame has an 33054f38fcaSMauro Carvalho Chehaberror, then all following decoded frames that refer to it also have the 33154f38fcaSMauro Carvalho Chehab``V4L2_BUF_FLAG_ERROR`` flag set, although the decoder will still try to 33254f38fcaSMauro Carvalho Chehabproduce (likely corrupted) frames. 33354f38fcaSMauro Carvalho Chehab 33454f38fcaSMauro Carvalho ChehabBuffer management while decoding 33554f38fcaSMauro Carvalho Chehab================================ 33654f38fcaSMauro Carvalho ChehabContrary to stateful decoders, a stateless decoder does not perform any kind of 33754f38fcaSMauro Carvalho Chehabbuffer management: it only guarantees that dequeued ``CAPTURE`` buffers can be 33854f38fcaSMauro Carvalho Chehabused by the client for as long as they are not queued again. "Used" here 33954f38fcaSMauro Carvalho Chehabencompasses using the buffer for compositing or display. 34054f38fcaSMauro Carvalho Chehab 34154f38fcaSMauro Carvalho ChehabA dequeued capture buffer can also be used as the reference frame of another 34254f38fcaSMauro Carvalho Chehabbuffer. 34354f38fcaSMauro Carvalho Chehab 34454f38fcaSMauro Carvalho ChehabA frame is specified as reference by converting its timestamp into nanoseconds, 34554f38fcaSMauro Carvalho Chehaband storing it into the relevant member of a codec-dependent control structure. 34654f38fcaSMauro Carvalho ChehabThe :c:func:`v4l2_timeval_to_ns` function must be used to perform that 34754f38fcaSMauro Carvalho Chehabconversion. The timestamp of a frame can be used to reference it as soon as all 34854f38fcaSMauro Carvalho Chehabits units of encoded data are successfully submitted to the ``OUTPUT`` queue. 34954f38fcaSMauro Carvalho Chehab 35054f38fcaSMauro Carvalho ChehabA decoded buffer containing a reference frame must not be reused as a decoding 35154f38fcaSMauro Carvalho Chehabtarget until all the frames referencing it have been decoded. The safest way to 35254f38fcaSMauro Carvalho Chehabachieve this is to refrain from queueing a reference buffer until all the 35354f38fcaSMauro Carvalho Chehabdecoded frames referencing it have been dequeued. However, if the driver can 35454f38fcaSMauro Carvalho Chehabguarantee that buffers queued to the ``CAPTURE`` queue are processed in queued 35554f38fcaSMauro Carvalho Chehaborder, then user-space can take advantage of this guarantee and queue a 35654f38fcaSMauro Carvalho Chehabreference buffer when the following conditions are met: 35754f38fcaSMauro Carvalho Chehab 35854f38fcaSMauro Carvalho Chehab1. All the requests for frames affected by the reference frame have been 35954f38fcaSMauro Carvalho Chehab queued, and 36054f38fcaSMauro Carvalho Chehab 36154f38fcaSMauro Carvalho Chehab2. A sufficient number of ``CAPTURE`` buffers to cover all the decoded 36254f38fcaSMauro Carvalho Chehab referencing frames have been queued. 36354f38fcaSMauro Carvalho Chehab 36454f38fcaSMauro Carvalho ChehabWhen queuing a decoding request, the driver will increase the reference count of 36554f38fcaSMauro Carvalho Chehaball the resources associated with reference frames. This means that the client 36654f38fcaSMauro Carvalho Chehabcan e.g. close the DMABUF file descriptors of reference frame buffers if it 36754f38fcaSMauro Carvalho Chehabwon't need them afterwards. 36854f38fcaSMauro Carvalho Chehab 36954f38fcaSMauro Carvalho ChehabSeeking 37054f38fcaSMauro Carvalho Chehab======= 37154f38fcaSMauro Carvalho ChehabIn order to seek, the client just needs to submit requests using input buffers 37254f38fcaSMauro Carvalho Chehabcorresponding to the new stream position. It must however be aware that 37354f38fcaSMauro Carvalho Chehabresolution may have changed and follow the dynamic resolution change sequence in 37454f38fcaSMauro Carvalho Chehabthat case. Also depending on the codec used, picture parameters (e.g. SPS/PPS 37554f38fcaSMauro Carvalho Chehabfor H.264) may have changed and the client is responsible for making sure that a 37654f38fcaSMauro Carvalho Chehabvalid state is sent to the decoder. 37754f38fcaSMauro Carvalho Chehab 37854f38fcaSMauro Carvalho ChehabThe client is then free to ignore any returned ``CAPTURE`` buffer that comes 37954f38fcaSMauro Carvalho Chehabfrom the pre-seek position. 38054f38fcaSMauro Carvalho Chehab 38154f38fcaSMauro Carvalho ChehabPausing 38254f38fcaSMauro Carvalho Chehab======= 38354f38fcaSMauro Carvalho Chehab 38454f38fcaSMauro Carvalho ChehabIn order to pause, the client can just cease queuing buffers onto the ``OUTPUT`` 38554f38fcaSMauro Carvalho Chehabqueue. Without source bytestream data, there is no data to process and the codec 38654f38fcaSMauro Carvalho Chehabwill remain idle. 38754f38fcaSMauro Carvalho Chehab 38854f38fcaSMauro Carvalho ChehabDynamic resolution change 38954f38fcaSMauro Carvalho Chehab========================= 39054f38fcaSMauro Carvalho Chehab 39154f38fcaSMauro Carvalho ChehabIf the client detects a resolution change in the stream, it will need to perform 39254f38fcaSMauro Carvalho Chehabthe initialization sequence again with the new resolution: 39354f38fcaSMauro Carvalho Chehab 39454f38fcaSMauro Carvalho Chehab1. If the last submitted request resulted in a ``CAPTURE`` buffer being 39554f38fcaSMauro Carvalho Chehab held by the use of the ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag, then the 39654f38fcaSMauro Carvalho Chehab last frame is not available on the ``CAPTURE`` queue. In this case, a 39754f38fcaSMauro Carvalho Chehab ``V4L2_DEC_CMD_FLUSH`` command shall be sent. This will make the driver 39854f38fcaSMauro Carvalho Chehab dequeue the held ``CAPTURE`` buffer. 39954f38fcaSMauro Carvalho Chehab 40054f38fcaSMauro Carvalho Chehab2. Wait until all submitted requests have completed and dequeue the 40154f38fcaSMauro Carvalho Chehab corresponding output buffers. 40254f38fcaSMauro Carvalho Chehab 40354f38fcaSMauro Carvalho Chehab3. Call :c:func:`VIDIOC_STREAMOFF` on both the ``OUTPUT`` and ``CAPTURE`` 40454f38fcaSMauro Carvalho Chehab queues. 40554f38fcaSMauro Carvalho Chehab 40654f38fcaSMauro Carvalho Chehab4. Free all ``CAPTURE`` buffers by calling :c:func:`VIDIOC_REQBUFS` on the 40754f38fcaSMauro Carvalho Chehab ``CAPTURE`` queue with a buffer count of zero. 40854f38fcaSMauro Carvalho Chehab 40954f38fcaSMauro Carvalho Chehab5. Perform the initialization sequence again (minus the allocation of 41054f38fcaSMauro Carvalho Chehab ``OUTPUT`` buffers), with the new resolution set on the ``OUTPUT`` queue. 41154f38fcaSMauro Carvalho Chehab Note that due to resolution constraints, a different format may need to be 41254f38fcaSMauro Carvalho Chehab picked on the ``CAPTURE`` queue. 41354f38fcaSMauro Carvalho Chehab 41454f38fcaSMauro Carvalho ChehabDrain 41554f38fcaSMauro Carvalho Chehab===== 41654f38fcaSMauro Carvalho Chehab 41754f38fcaSMauro Carvalho ChehabIf the last submitted request resulted in a ``CAPTURE`` buffer being 41854f38fcaSMauro Carvalho Chehabheld by the use of the ``V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF`` flag, then the 41954f38fcaSMauro Carvalho Chehablast frame is not available on the ``CAPTURE`` queue. In this case, a 42054f38fcaSMauro Carvalho Chehab``V4L2_DEC_CMD_FLUSH`` command shall be sent. This will make the driver 42154f38fcaSMauro Carvalho Chehabdequeue the held ``CAPTURE`` buffer. 42254f38fcaSMauro Carvalho Chehab 42354f38fcaSMauro Carvalho ChehabAfter that, in order to drain the stream on a stateless decoder, the client 42454f38fcaSMauro Carvalho Chehabjust needs to wait until all the submitted requests are completed. 425