154f38fcaSMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0 254f38fcaSMauro Carvalho Chehab 354f38fcaSMauro Carvalho Chehab.. _decoder: 454f38fcaSMauro Carvalho Chehab 554f38fcaSMauro Carvalho Chehab************************************************* 654f38fcaSMauro Carvalho ChehabMemory-to-Memory Stateful Video Decoder Interface 754f38fcaSMauro Carvalho Chehab************************************************* 854f38fcaSMauro Carvalho Chehab 954f38fcaSMauro Carvalho ChehabA stateful video decoder takes complete chunks of the bytestream (e.g. Annex-B 1054f38fcaSMauro Carvalho ChehabH.264/HEVC stream, raw VP8/9 stream) and decodes them into raw video frames in 1154f38fcaSMauro Carvalho Chehabdisplay order. The decoder is expected not to require any additional information 1254f38fcaSMauro Carvalho Chehabfrom the client to process these buffers. 1354f38fcaSMauro Carvalho Chehab 1454f38fcaSMauro Carvalho ChehabPerforming software parsing, processing etc. of the stream in the driver in 1554f38fcaSMauro Carvalho Chehaborder to support this interface is strongly discouraged. In case such 1654f38fcaSMauro Carvalho Chehaboperations are needed, use of the Stateless Video Decoder Interface (in 1754f38fcaSMauro Carvalho Chehabdevelopment) is strongly advised. 1854f38fcaSMauro Carvalho Chehab 1954f38fcaSMauro Carvalho ChehabConventions and Notations Used in This Document 2054f38fcaSMauro Carvalho Chehab=============================================== 2154f38fcaSMauro Carvalho Chehab 2254f38fcaSMauro Carvalho Chehab1. The general V4L2 API rules apply if not specified in this document 2354f38fcaSMauro Carvalho Chehab otherwise. 2454f38fcaSMauro Carvalho Chehab 2554f38fcaSMauro Carvalho Chehab2. The meaning of words "must", "may", "should", etc. is as per `RFC 2654f38fcaSMauro Carvalho Chehab 2119 <https://tools.ietf.org/html/rfc2119>`_. 2754f38fcaSMauro Carvalho Chehab 2854f38fcaSMauro Carvalho Chehab3. All steps not marked "optional" are required. 2954f38fcaSMauro Carvalho Chehab 3054f38fcaSMauro Carvalho Chehab4. :c:func:`VIDIOC_G_EXT_CTRLS` and :c:func:`VIDIOC_S_EXT_CTRLS` may be used 3154f38fcaSMauro Carvalho Chehab interchangeably with :c:func:`VIDIOC_G_CTRL` and :c:func:`VIDIOC_S_CTRL`, 3254f38fcaSMauro Carvalho Chehab unless specified otherwise. 3354f38fcaSMauro Carvalho Chehab 3454f38fcaSMauro Carvalho Chehab5. Single-planar API (see :ref:`planar-apis`) and applicable structures may be 3554f38fcaSMauro Carvalho Chehab used interchangeably with multi-planar API, unless specified otherwise, 3654f38fcaSMauro Carvalho Chehab depending on decoder capabilities and following the general V4L2 guidelines. 3754f38fcaSMauro Carvalho Chehab 3854f38fcaSMauro Carvalho Chehab6. i = [a..b]: sequence of integers from a to b, inclusive, i.e. i = 3954f38fcaSMauro Carvalho Chehab [0..2]: i = 0, 1, 2. 4054f38fcaSMauro Carvalho Chehab 41eff7d26aSMauro Carvalho Chehab7. Given an ``OUTPUT`` buffer A, then A' represents a buffer on the ``CAPTURE`` 4254f38fcaSMauro Carvalho Chehab queue containing data that resulted from processing buffer A. 4354f38fcaSMauro Carvalho Chehab 4454f38fcaSMauro Carvalho Chehab.. _decoder-glossary: 4554f38fcaSMauro Carvalho Chehab 4654f38fcaSMauro Carvalho ChehabGlossary 4754f38fcaSMauro Carvalho Chehab======== 4854f38fcaSMauro Carvalho Chehab 4954f38fcaSMauro Carvalho ChehabCAPTURE 5054f38fcaSMauro Carvalho Chehab the destination buffer queue; for decoders, the queue of buffers containing 5154f38fcaSMauro Carvalho Chehab decoded frames; for encoders, the queue of buffers containing an encoded 5254f38fcaSMauro Carvalho Chehab bytestream; ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` or 5354f38fcaSMauro Carvalho Chehab ``V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE``; data is captured from the hardware 5454f38fcaSMauro Carvalho Chehab into ``CAPTURE`` buffers. 5554f38fcaSMauro Carvalho Chehab 5654f38fcaSMauro Carvalho Chehabclient 5754f38fcaSMauro Carvalho Chehab the application communicating with the decoder or encoder implementing 5854f38fcaSMauro Carvalho Chehab this interface. 5954f38fcaSMauro Carvalho Chehab 6054f38fcaSMauro Carvalho Chehabcoded format 6154f38fcaSMauro Carvalho Chehab encoded/compressed video bytestream format (e.g. H.264, VP8, etc.); see 6254f38fcaSMauro Carvalho Chehab also: raw format. 6354f38fcaSMauro Carvalho Chehab 6454f38fcaSMauro Carvalho Chehabcoded height 6554f38fcaSMauro Carvalho Chehab height for given coded resolution. 6654f38fcaSMauro Carvalho Chehab 6754f38fcaSMauro Carvalho Chehabcoded resolution 6854f38fcaSMauro Carvalho Chehab stream resolution in pixels aligned to codec and hardware requirements; 6954f38fcaSMauro Carvalho Chehab typically visible resolution rounded up to full macroblocks; 7054f38fcaSMauro Carvalho Chehab see also: visible resolution. 7154f38fcaSMauro Carvalho Chehab 7254f38fcaSMauro Carvalho Chehabcoded width 7354f38fcaSMauro Carvalho Chehab width for given coded resolution. 7454f38fcaSMauro Carvalho Chehab 7587581a9eSSebastian Frickecoding tree unit 7687581a9eSSebastian Fricke processing unit of the HEVC codec (corresponds to macroblock units in 7787581a9eSSebastian Fricke H.264, VP8, VP9), 7887581a9eSSebastian Fricke can use block structures of up to 64×64 pixels. 7987581a9eSSebastian Fricke Good at sub-partitioning the picture into variable sized structures. 8087581a9eSSebastian Fricke 8154f38fcaSMauro Carvalho Chehabdecode order 8254f38fcaSMauro Carvalho Chehab the order in which frames are decoded; may differ from display order if the 8354f38fcaSMauro Carvalho Chehab coded format includes a feature of frame reordering; for decoders, 8454f38fcaSMauro Carvalho Chehab ``OUTPUT`` buffers must be queued by the client in decode order; for 8554f38fcaSMauro Carvalho Chehab encoders ``CAPTURE`` buffers must be returned by the encoder in decode order. 8654f38fcaSMauro Carvalho Chehab 8754f38fcaSMauro Carvalho Chehabdestination 8854f38fcaSMauro Carvalho Chehab data resulting from the decode process; see ``CAPTURE``. 8954f38fcaSMauro Carvalho Chehab 9054f38fcaSMauro Carvalho Chehabdisplay order 9154f38fcaSMauro Carvalho Chehab the order in which frames must be displayed; for encoders, ``OUTPUT`` 9254f38fcaSMauro Carvalho Chehab buffers must be queued by the client in display order; for decoders, 9354f38fcaSMauro Carvalho Chehab ``CAPTURE`` buffers must be returned by the decoder in display order. 9454f38fcaSMauro Carvalho Chehab 9554f38fcaSMauro Carvalho ChehabDPB 9654f38fcaSMauro Carvalho Chehab Decoded Picture Buffer; an H.264/HEVC term for a buffer that stores a decoded 9754f38fcaSMauro Carvalho Chehab raw frame available for reference in further decoding steps. 9854f38fcaSMauro Carvalho Chehab 9954f38fcaSMauro Carvalho ChehabEOS 10054f38fcaSMauro Carvalho Chehab end of stream. 10154f38fcaSMauro Carvalho Chehab 10254f38fcaSMauro Carvalho ChehabIDR 10354f38fcaSMauro Carvalho Chehab Instantaneous Decoder Refresh; a type of a keyframe in an H.264/HEVC-encoded 10454f38fcaSMauro Carvalho Chehab stream, which clears the list of earlier reference frames (DPBs). 10554f38fcaSMauro Carvalho Chehab 10654f38fcaSMauro Carvalho Chehabkeyframe 10754f38fcaSMauro Carvalho Chehab an encoded frame that does not reference frames decoded earlier, i.e. 10854f38fcaSMauro Carvalho Chehab can be decoded fully on its own. 10954f38fcaSMauro Carvalho Chehab 11054f38fcaSMauro Carvalho Chehabmacroblock 11154f38fcaSMauro Carvalho Chehab a processing unit in image and video compression formats based on linear 11254f38fcaSMauro Carvalho Chehab block transforms (e.g. H.264, VP8, VP9); codec-specific, but for most of 11387581a9eSSebastian Fricke popular codecs the size is 16x16 samples (pixels). The HEVC codec uses a 11487581a9eSSebastian Fricke slightly more flexible processing unit called coding tree unit (CTU). 11554f38fcaSMauro Carvalho Chehab 11654f38fcaSMauro Carvalho ChehabOUTPUT 11754f38fcaSMauro Carvalho Chehab the source buffer queue; for decoders, the queue of buffers containing 11854f38fcaSMauro Carvalho Chehab an encoded bytestream; for encoders, the queue of buffers containing raw 11954f38fcaSMauro Carvalho Chehab frames; ``V4L2_BUF_TYPE_VIDEO_OUTPUT`` or 12054f38fcaSMauro Carvalho Chehab ``V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE``; the hardware is fed with data 12154f38fcaSMauro Carvalho Chehab from ``OUTPUT`` buffers. 12254f38fcaSMauro Carvalho Chehab 12354f38fcaSMauro Carvalho ChehabPPS 12454f38fcaSMauro Carvalho Chehab Picture Parameter Set; a type of metadata entity in an H.264/HEVC bytestream. 12554f38fcaSMauro Carvalho Chehab 12654f38fcaSMauro Carvalho Chehabraw format 12754f38fcaSMauro Carvalho Chehab uncompressed format containing raw pixel data (e.g. YUV, RGB formats). 12854f38fcaSMauro Carvalho Chehab 12954f38fcaSMauro Carvalho Chehabresume point 13054f38fcaSMauro Carvalho Chehab a point in the bytestream from which decoding may start/continue, without 13154f38fcaSMauro Carvalho Chehab any previous state/data present, e.g.: a keyframe (VP8/VP9) or 13254f38fcaSMauro Carvalho Chehab SPS/PPS/IDR sequence (H.264/HEVC); a resume point is required to start decode 13354f38fcaSMauro Carvalho Chehab of a new stream, or to resume decoding after a seek. 13454f38fcaSMauro Carvalho Chehab 13554f38fcaSMauro Carvalho Chehabsource 13654f38fcaSMauro Carvalho Chehab data fed to the decoder or encoder; see ``OUTPUT``. 13754f38fcaSMauro Carvalho Chehab 13854f38fcaSMauro Carvalho Chehabsource height 13954f38fcaSMauro Carvalho Chehab height in pixels for given source resolution; relevant to encoders only. 14054f38fcaSMauro Carvalho Chehab 14154f38fcaSMauro Carvalho Chehabsource resolution 14254f38fcaSMauro Carvalho Chehab resolution in pixels of source frames being source to the encoder and 14354f38fcaSMauro Carvalho Chehab subject to further cropping to the bounds of visible resolution; relevant to 14454f38fcaSMauro Carvalho Chehab encoders only. 14554f38fcaSMauro Carvalho Chehab 14654f38fcaSMauro Carvalho Chehabsource width 14754f38fcaSMauro Carvalho Chehab width in pixels for given source resolution; relevant to encoders only. 14854f38fcaSMauro Carvalho Chehab 14954f38fcaSMauro Carvalho ChehabSPS 15054f38fcaSMauro Carvalho Chehab Sequence Parameter Set; a type of metadata entity in an H.264/HEVC bytestream. 15154f38fcaSMauro Carvalho Chehab 15254f38fcaSMauro Carvalho Chehabstream metadata 15354f38fcaSMauro Carvalho Chehab additional (non-visual) information contained inside encoded bytestream; 15454f38fcaSMauro Carvalho Chehab for example: coded resolution, visible resolution, codec profile. 15554f38fcaSMauro Carvalho Chehab 15654f38fcaSMauro Carvalho Chehabvisible height 15754f38fcaSMauro Carvalho Chehab height for given visible resolution; display height. 15854f38fcaSMauro Carvalho Chehab 15954f38fcaSMauro Carvalho Chehabvisible resolution 16054f38fcaSMauro Carvalho Chehab stream resolution of the visible picture, in pixels, to be used for 16154f38fcaSMauro Carvalho Chehab display purposes; must be smaller or equal to coded resolution; 16254f38fcaSMauro Carvalho Chehab display resolution. 16354f38fcaSMauro Carvalho Chehab 16454f38fcaSMauro Carvalho Chehabvisible width 16554f38fcaSMauro Carvalho Chehab width for given visible resolution; display width. 16654f38fcaSMauro Carvalho Chehab 16754f38fcaSMauro Carvalho ChehabState Machine 16854f38fcaSMauro Carvalho Chehab============= 16954f38fcaSMauro Carvalho Chehab 17054f38fcaSMauro Carvalho Chehab.. kernel-render:: DOT 17154f38fcaSMauro Carvalho Chehab :alt: DOT digraph of decoder state machine 17254f38fcaSMauro Carvalho Chehab :caption: Decoder State Machine 17354f38fcaSMauro Carvalho Chehab 17454f38fcaSMauro Carvalho Chehab digraph decoder_state_machine { 17554f38fcaSMauro Carvalho Chehab node [shape = doublecircle, label="Decoding"] Decoding; 17654f38fcaSMauro Carvalho Chehab 17754f38fcaSMauro Carvalho Chehab node [shape = circle, label="Initialization"] Initialization; 17854f38fcaSMauro Carvalho Chehab node [shape = circle, label="Capture\nsetup"] CaptureSetup; 17954f38fcaSMauro Carvalho Chehab node [shape = circle, label="Dynamic\nResolution\nChange"] ResChange; 18054f38fcaSMauro Carvalho Chehab node [shape = circle, label="Stopped"] Stopped; 18154f38fcaSMauro Carvalho Chehab node [shape = circle, label="Drain"] Drain; 18254f38fcaSMauro Carvalho Chehab node [shape = circle, label="Seek"] Seek; 18354f38fcaSMauro Carvalho Chehab node [shape = circle, label="End of Stream"] EoS; 18454f38fcaSMauro Carvalho Chehab 18554f38fcaSMauro Carvalho Chehab node [shape = point]; qi 18654f38fcaSMauro Carvalho Chehab qi -> Initialization [ label = "open()" ]; 18754f38fcaSMauro Carvalho Chehab 18854f38fcaSMauro Carvalho Chehab Initialization -> CaptureSetup [ label = "CAPTURE\nformat\nestablished" ]; 18954f38fcaSMauro Carvalho Chehab 19054f38fcaSMauro Carvalho Chehab CaptureSetup -> Stopped [ label = "CAPTURE\nbuffers\nready" ]; 19154f38fcaSMauro Carvalho Chehab 19254f38fcaSMauro Carvalho Chehab Decoding -> ResChange [ label = "Stream\nresolution\nchange" ]; 19354f38fcaSMauro Carvalho Chehab Decoding -> Drain [ label = "V4L2_DEC_CMD_STOP" ]; 19454f38fcaSMauro Carvalho Chehab Decoding -> EoS [ label = "EoS mark\nin the stream" ]; 19554f38fcaSMauro Carvalho Chehab Decoding -> Seek [ label = "VIDIOC_STREAMOFF(OUTPUT)" ]; 19654f38fcaSMauro Carvalho Chehab Decoding -> Stopped [ label = "VIDIOC_STREAMOFF(CAPTURE)" ]; 19754f38fcaSMauro Carvalho Chehab Decoding -> Decoding; 19854f38fcaSMauro Carvalho Chehab 19954f38fcaSMauro Carvalho Chehab ResChange -> CaptureSetup [ label = "CAPTURE\nformat\nestablished" ]; 20054f38fcaSMauro Carvalho Chehab ResChange -> Seek [ label = "VIDIOC_STREAMOFF(OUTPUT)" ]; 20154f38fcaSMauro Carvalho Chehab 20254f38fcaSMauro Carvalho Chehab EoS -> Drain [ label = "Implicit\ndrain" ]; 20354f38fcaSMauro Carvalho Chehab 20454f38fcaSMauro Carvalho Chehab Drain -> Stopped [ label = "All CAPTURE\nbuffers dequeued\nor\nVIDIOC_STREAMOFF(CAPTURE)" ]; 20554f38fcaSMauro Carvalho Chehab Drain -> Seek [ label = "VIDIOC_STREAMOFF(OUTPUT)" ]; 20654f38fcaSMauro Carvalho Chehab 20754f38fcaSMauro Carvalho Chehab Seek -> Decoding [ label = "VIDIOC_STREAMON(OUTPUT)" ]; 20854f38fcaSMauro Carvalho Chehab Seek -> Initialization [ label = "VIDIOC_REQBUFS(OUTPUT, 0)" ]; 20954f38fcaSMauro Carvalho Chehab 21054f38fcaSMauro Carvalho Chehab Stopped -> Decoding [ label = "V4L2_DEC_CMD_START\nor\nVIDIOC_STREAMON(CAPTURE)" ]; 21154f38fcaSMauro Carvalho Chehab Stopped -> Seek [ label = "VIDIOC_STREAMOFF(OUTPUT)" ]; 21254f38fcaSMauro Carvalho Chehab } 21354f38fcaSMauro Carvalho Chehab 21454f38fcaSMauro Carvalho ChehabQuerying Capabilities 21554f38fcaSMauro Carvalho Chehab===================== 21654f38fcaSMauro Carvalho Chehab 21754f38fcaSMauro Carvalho Chehab1. To enumerate the set of coded formats supported by the decoder, the 21854f38fcaSMauro Carvalho Chehab client may call :c:func:`VIDIOC_ENUM_FMT` on ``OUTPUT``. 21954f38fcaSMauro Carvalho Chehab 22054f38fcaSMauro Carvalho Chehab * The full set of supported formats will be returned, regardless of the 22154f38fcaSMauro Carvalho Chehab format set on ``CAPTURE``. 22254f38fcaSMauro Carvalho Chehab * Check the flags field of :c:type:`v4l2_fmtdesc` for more information 22354f38fcaSMauro Carvalho Chehab about the decoder's capabilities with respect to each coded format. 22454f38fcaSMauro Carvalho Chehab In particular whether or not the decoder has a full-fledged bytestream 22554f38fcaSMauro Carvalho Chehab parser and if the decoder supports dynamic resolution changes. 22654f38fcaSMauro Carvalho Chehab 22754f38fcaSMauro Carvalho Chehab2. To enumerate the set of supported raw formats, the client may call 22854f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_ENUM_FMT` on ``CAPTURE``. 22954f38fcaSMauro Carvalho Chehab 23054f38fcaSMauro Carvalho Chehab * Only the formats supported for the format currently active on ``OUTPUT`` 23154f38fcaSMauro Carvalho Chehab will be returned. 23254f38fcaSMauro Carvalho Chehab 23354f38fcaSMauro Carvalho Chehab * In order to enumerate raw formats supported by a given coded format, 23454f38fcaSMauro Carvalho Chehab the client must first set that coded format on ``OUTPUT`` and then 23554f38fcaSMauro Carvalho Chehab enumerate formats on ``CAPTURE``. 23654f38fcaSMauro Carvalho Chehab 23754f38fcaSMauro Carvalho Chehab3. The client may use :c:func:`VIDIOC_ENUM_FRAMESIZES` to detect supported 23854f38fcaSMauro Carvalho Chehab resolutions for a given format, passing desired pixel format in 23954f38fcaSMauro Carvalho Chehab :c:type:`v4l2_frmsizeenum` ``pixel_format``. 24054f38fcaSMauro Carvalho Chehab 24154f38fcaSMauro Carvalho Chehab * Values returned by :c:func:`VIDIOC_ENUM_FRAMESIZES` for a coded pixel 24254f38fcaSMauro Carvalho Chehab format will include all possible coded resolutions supported by the 24354f38fcaSMauro Carvalho Chehab decoder for given coded pixel format. 24454f38fcaSMauro Carvalho Chehab 24554f38fcaSMauro Carvalho Chehab * Values returned by :c:func:`VIDIOC_ENUM_FRAMESIZES` for a raw pixel format 24654f38fcaSMauro Carvalho Chehab will include all possible frame buffer resolutions supported by the 24754f38fcaSMauro Carvalho Chehab decoder for given raw pixel format and the coded format currently set on 24854f38fcaSMauro Carvalho Chehab ``OUTPUT``. 24954f38fcaSMauro Carvalho Chehab 25054f38fcaSMauro Carvalho Chehab4. Supported profiles and levels for the coded format currently set on 25154f38fcaSMauro Carvalho Chehab ``OUTPUT``, if applicable, may be queried using their respective controls 25254f38fcaSMauro Carvalho Chehab via :c:func:`VIDIOC_QUERYCTRL`. 25354f38fcaSMauro Carvalho Chehab 25454f38fcaSMauro Carvalho ChehabInitialization 25554f38fcaSMauro Carvalho Chehab============== 25654f38fcaSMauro Carvalho Chehab 257eef7a5e7SHans Verkuil1. Set the coded format on ``OUTPUT`` via :c:func:`VIDIOC_S_FMT`. 25854f38fcaSMauro Carvalho Chehab 25954f38fcaSMauro Carvalho Chehab * **Required fields:** 26054f38fcaSMauro Carvalho Chehab 26154f38fcaSMauro Carvalho Chehab ``type`` 26254f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 26354f38fcaSMauro Carvalho Chehab 26454f38fcaSMauro Carvalho Chehab ``pixelformat`` 26554f38fcaSMauro Carvalho Chehab a coded pixel format. 26654f38fcaSMauro Carvalho Chehab 26754f38fcaSMauro Carvalho Chehab ``width``, ``height`` 26854f38fcaSMauro Carvalho Chehab coded resolution of the stream; required only if it cannot be parsed 26954f38fcaSMauro Carvalho Chehab from the stream for the given coded format; otherwise the decoder will 27054f38fcaSMauro Carvalho Chehab use this resolution as a placeholder resolution that will likely change 27154f38fcaSMauro Carvalho Chehab as soon as it can parse the actual coded resolution from the stream. 27254f38fcaSMauro Carvalho Chehab 27354f38fcaSMauro Carvalho Chehab ``sizeimage`` 27454f38fcaSMauro Carvalho Chehab desired size of ``OUTPUT`` buffers; the decoder may adjust it to 27554f38fcaSMauro Carvalho Chehab match hardware requirements. 27654f38fcaSMauro Carvalho Chehab 27754f38fcaSMauro Carvalho Chehab other fields 27854f38fcaSMauro Carvalho Chehab follow standard semantics. 27954f38fcaSMauro Carvalho Chehab 280*1073f441SPaul Kocialkowski * **Returned fields:** 28154f38fcaSMauro Carvalho Chehab 28254f38fcaSMauro Carvalho Chehab ``sizeimage`` 28354f38fcaSMauro Carvalho Chehab adjusted size of ``OUTPUT`` buffers. 28454f38fcaSMauro Carvalho Chehab 28554f38fcaSMauro Carvalho Chehab * The ``CAPTURE`` format will be updated with an appropriate frame buffer 28654f38fcaSMauro Carvalho Chehab resolution instantly based on the width and height returned by 28754f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_S_FMT`. 28854f38fcaSMauro Carvalho Chehab However, for coded formats that include stream resolution information, 28954f38fcaSMauro Carvalho Chehab after the decoder is done parsing the information from the stream, it will 29054f38fcaSMauro Carvalho Chehab update the ``CAPTURE`` format with new values and signal a source change 29154f38fcaSMauro Carvalho Chehab event, regardless of whether they match the values set by the client or 29254f38fcaSMauro Carvalho Chehab not. 29354f38fcaSMauro Carvalho Chehab 29454f38fcaSMauro Carvalho Chehab .. important:: 29554f38fcaSMauro Carvalho Chehab 29654f38fcaSMauro Carvalho Chehab Changing the ``OUTPUT`` format may change the currently set ``CAPTURE`` 29754f38fcaSMauro Carvalho Chehab format. How the new ``CAPTURE`` format is determined is up to the decoder 298eff7d26aSMauro Carvalho Chehab and the client must ensure it matches its needs afterwards. 29954f38fcaSMauro Carvalho Chehab 30054f38fcaSMauro Carvalho Chehab2. Allocate source (bytestream) buffers via :c:func:`VIDIOC_REQBUFS` on 30154f38fcaSMauro Carvalho Chehab ``OUTPUT``. 30254f38fcaSMauro Carvalho Chehab 30354f38fcaSMauro Carvalho Chehab * **Required fields:** 30454f38fcaSMauro Carvalho Chehab 30554f38fcaSMauro Carvalho Chehab ``count`` 30654f38fcaSMauro Carvalho Chehab requested number of buffers to allocate; greater than zero. 30754f38fcaSMauro Carvalho Chehab 30854f38fcaSMauro Carvalho Chehab ``type`` 30954f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 31054f38fcaSMauro Carvalho Chehab 31154f38fcaSMauro Carvalho Chehab ``memory`` 31254f38fcaSMauro Carvalho Chehab follows standard semantics. 31354f38fcaSMauro Carvalho Chehab 314*1073f441SPaul Kocialkowski * **Returned fields:** 31554f38fcaSMauro Carvalho Chehab 31654f38fcaSMauro Carvalho Chehab ``count`` 31754f38fcaSMauro Carvalho Chehab the actual number of buffers allocated. 31854f38fcaSMauro Carvalho Chehab 31954f38fcaSMauro Carvalho Chehab .. warning:: 32054f38fcaSMauro Carvalho Chehab 32154f38fcaSMauro Carvalho Chehab The actual number of allocated buffers may differ from the ``count`` 32254f38fcaSMauro Carvalho Chehab given. The client must check the updated value of ``count`` after the 32354f38fcaSMauro Carvalho Chehab call returns. 32454f38fcaSMauro Carvalho Chehab 32554f38fcaSMauro Carvalho Chehab Alternatively, :c:func:`VIDIOC_CREATE_BUFS` on the ``OUTPUT`` queue can be 32654f38fcaSMauro Carvalho Chehab used to have more control over buffer allocation. 32754f38fcaSMauro Carvalho Chehab 32854f38fcaSMauro Carvalho Chehab * **Required fields:** 32954f38fcaSMauro Carvalho Chehab 33054f38fcaSMauro Carvalho Chehab ``count`` 33154f38fcaSMauro Carvalho Chehab requested number of buffers to allocate; greater than zero. 33254f38fcaSMauro Carvalho Chehab 33354f38fcaSMauro Carvalho Chehab ``type`` 33454f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 33554f38fcaSMauro Carvalho Chehab 33654f38fcaSMauro Carvalho Chehab ``memory`` 33754f38fcaSMauro Carvalho Chehab follows standard semantics. 33854f38fcaSMauro Carvalho Chehab 33954f38fcaSMauro Carvalho Chehab ``format`` 34054f38fcaSMauro Carvalho Chehab follows standard semantics. 34154f38fcaSMauro Carvalho Chehab 342*1073f441SPaul Kocialkowski * **Returned fields:** 34354f38fcaSMauro Carvalho Chehab 34454f38fcaSMauro Carvalho Chehab ``count`` 34554f38fcaSMauro Carvalho Chehab adjusted to the number of allocated buffers. 34654f38fcaSMauro Carvalho Chehab 34754f38fcaSMauro Carvalho Chehab .. warning:: 34854f38fcaSMauro Carvalho Chehab 34954f38fcaSMauro Carvalho Chehab The actual number of allocated buffers may differ from the ``count`` 35054f38fcaSMauro Carvalho Chehab given. The client must check the updated value of ``count`` after the 35154f38fcaSMauro Carvalho Chehab call returns. 35254f38fcaSMauro Carvalho Chehab 35354f38fcaSMauro Carvalho Chehab3. Start streaming on the ``OUTPUT`` queue via :c:func:`VIDIOC_STREAMON`. 35454f38fcaSMauro Carvalho Chehab 35554f38fcaSMauro Carvalho Chehab4. **This step only applies to coded formats that contain resolution information 35654f38fcaSMauro Carvalho Chehab in the stream.** Continue queuing/dequeuing bytestream buffers to/from the 35754f38fcaSMauro Carvalho Chehab ``OUTPUT`` queue via :c:func:`VIDIOC_QBUF` and :c:func:`VIDIOC_DQBUF`. The 35854f38fcaSMauro Carvalho Chehab buffers will be processed and returned to the client in order, until 35954f38fcaSMauro Carvalho Chehab required metadata to configure the ``CAPTURE`` queue are found. This is 36054f38fcaSMauro Carvalho Chehab indicated by the decoder sending a ``V4L2_EVENT_SOURCE_CHANGE`` event with 36154f38fcaSMauro Carvalho Chehab ``changes`` set to ``V4L2_EVENT_SRC_CH_RESOLUTION``. 36254f38fcaSMauro Carvalho Chehab 36354f38fcaSMauro Carvalho Chehab * It is not an error if the first buffer does not contain enough data for 36454f38fcaSMauro Carvalho Chehab this to occur. Processing of the buffers will continue as long as more 36554f38fcaSMauro Carvalho Chehab data is needed. 36654f38fcaSMauro Carvalho Chehab 36754f38fcaSMauro Carvalho Chehab * If data in a buffer that triggers the event is required to decode the 36854f38fcaSMauro Carvalho Chehab first frame, it will not be returned to the client, until the 36954f38fcaSMauro Carvalho Chehab initialization sequence completes and the frame is decoded. 37054f38fcaSMauro Carvalho Chehab 37154f38fcaSMauro Carvalho Chehab * If the client has not set the coded resolution of the stream on its own, 37254f38fcaSMauro Carvalho Chehab calling :c:func:`VIDIOC_G_FMT`, :c:func:`VIDIOC_S_FMT`, 37354f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_TRY_FMT` or :c:func:`VIDIOC_REQBUFS` on the ``CAPTURE`` 37454f38fcaSMauro Carvalho Chehab queue will not return the real values for the stream until a 37554f38fcaSMauro Carvalho Chehab ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to 37654f38fcaSMauro Carvalho Chehab ``V4L2_EVENT_SRC_CH_RESOLUTION`` is signaled. 37754f38fcaSMauro Carvalho Chehab 37854f38fcaSMauro Carvalho Chehab .. important:: 37954f38fcaSMauro Carvalho Chehab 38054f38fcaSMauro Carvalho Chehab Any client query issued after the decoder queues the event will return 38154f38fcaSMauro Carvalho Chehab values applying to the just parsed stream, including queue formats, 38254f38fcaSMauro Carvalho Chehab selection rectangles and controls. 38354f38fcaSMauro Carvalho Chehab 38454f38fcaSMauro Carvalho Chehab .. note:: 38554f38fcaSMauro Carvalho Chehab 38654f38fcaSMauro Carvalho Chehab A client capable of acquiring stream parameters from the bytestream on 38754f38fcaSMauro Carvalho Chehab its own may attempt to set the width and height of the ``OUTPUT`` format 38854f38fcaSMauro Carvalho Chehab to non-zero values matching the coded size of the stream, skip this step 38954f38fcaSMauro Carvalho Chehab and continue with the `Capture Setup` sequence. However, it must not 39054f38fcaSMauro Carvalho Chehab rely on any driver queries regarding stream parameters, such as 39154f38fcaSMauro Carvalho Chehab selection rectangles and controls, since the decoder has not parsed them 39254f38fcaSMauro Carvalho Chehab from the stream yet. If the values configured by the client do not match 39354f38fcaSMauro Carvalho Chehab those parsed by the decoder, a `Dynamic Resolution Change` will be 39454f38fcaSMauro Carvalho Chehab triggered to reconfigure them. 39554f38fcaSMauro Carvalho Chehab 39654f38fcaSMauro Carvalho Chehab .. note:: 39754f38fcaSMauro Carvalho Chehab 39854f38fcaSMauro Carvalho Chehab No decoded frames are produced during this phase. 39954f38fcaSMauro Carvalho Chehab 40054f38fcaSMauro Carvalho Chehab5. Continue with the `Capture Setup` sequence. 40154f38fcaSMauro Carvalho Chehab 40254f38fcaSMauro Carvalho ChehabCapture Setup 40354f38fcaSMauro Carvalho Chehab============= 40454f38fcaSMauro Carvalho Chehab 40554f38fcaSMauro Carvalho Chehab1. Call :c:func:`VIDIOC_G_FMT` on the ``CAPTURE`` queue to get format for the 40654f38fcaSMauro Carvalho Chehab destination buffers parsed/decoded from the bytestream. 40754f38fcaSMauro Carvalho Chehab 40854f38fcaSMauro Carvalho Chehab * **Required fields:** 40954f38fcaSMauro Carvalho Chehab 41054f38fcaSMauro Carvalho Chehab ``type`` 41154f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 41254f38fcaSMauro Carvalho Chehab 413*1073f441SPaul Kocialkowski * **Returned fields:** 41454f38fcaSMauro Carvalho Chehab 41554f38fcaSMauro Carvalho Chehab ``width``, ``height`` 41654f38fcaSMauro Carvalho Chehab frame buffer resolution for the decoded frames. 41754f38fcaSMauro Carvalho Chehab 41854f38fcaSMauro Carvalho Chehab ``pixelformat`` 41954f38fcaSMauro Carvalho Chehab pixel format for decoded frames. 42054f38fcaSMauro Carvalho Chehab 42154f38fcaSMauro Carvalho Chehab ``num_planes`` (for _MPLANE ``type`` only) 42254f38fcaSMauro Carvalho Chehab number of planes for pixelformat. 42354f38fcaSMauro Carvalho Chehab 42454f38fcaSMauro Carvalho Chehab ``sizeimage``, ``bytesperline`` 42554f38fcaSMauro Carvalho Chehab as per standard semantics; matching frame buffer format. 42654f38fcaSMauro Carvalho Chehab 42754f38fcaSMauro Carvalho Chehab .. note:: 42854f38fcaSMauro Carvalho Chehab 42954f38fcaSMauro Carvalho Chehab The value of ``pixelformat`` may be any pixel format supported by the 43054f38fcaSMauro Carvalho Chehab decoder for the current stream. The decoder should choose a 43154f38fcaSMauro Carvalho Chehab preferred/optimal format for the default configuration. For example, a 43254f38fcaSMauro Carvalho Chehab YUV format may be preferred over an RGB format if an additional 43354f38fcaSMauro Carvalho Chehab conversion step would be required for the latter. 43454f38fcaSMauro Carvalho Chehab 43554f38fcaSMauro Carvalho Chehab2. **Optional.** Acquire the visible resolution via 43654f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_G_SELECTION`. 43754f38fcaSMauro Carvalho Chehab 43854f38fcaSMauro Carvalho Chehab * **Required fields:** 43954f38fcaSMauro Carvalho Chehab 44054f38fcaSMauro Carvalho Chehab ``type`` 44154f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 44254f38fcaSMauro Carvalho Chehab 44354f38fcaSMauro Carvalho Chehab ``target`` 44454f38fcaSMauro Carvalho Chehab set to ``V4L2_SEL_TGT_COMPOSE``. 44554f38fcaSMauro Carvalho Chehab 446*1073f441SPaul Kocialkowski * **Returned fields:** 44754f38fcaSMauro Carvalho Chehab 44854f38fcaSMauro Carvalho Chehab ``r.left``, ``r.top``, ``r.width``, ``r.height`` 44954f38fcaSMauro Carvalho Chehab the visible rectangle; it must fit within the frame buffer resolution 45054f38fcaSMauro Carvalho Chehab returned by :c:func:`VIDIOC_G_FMT` on ``CAPTURE``. 45154f38fcaSMauro Carvalho Chehab 45254f38fcaSMauro Carvalho Chehab * The following selection targets are supported on ``CAPTURE``: 45354f38fcaSMauro Carvalho Chehab 45454f38fcaSMauro Carvalho Chehab ``V4L2_SEL_TGT_CROP_BOUNDS`` 45554f38fcaSMauro Carvalho Chehab corresponds to the coded resolution of the stream. 45654f38fcaSMauro Carvalho Chehab 45754f38fcaSMauro Carvalho Chehab ``V4L2_SEL_TGT_CROP_DEFAULT`` 45854f38fcaSMauro Carvalho Chehab the rectangle covering the part of the ``CAPTURE`` buffer that 45954f38fcaSMauro Carvalho Chehab contains meaningful picture data (visible area); width and height 46054f38fcaSMauro Carvalho Chehab will be equal to the visible resolution of the stream. 46154f38fcaSMauro Carvalho Chehab 46254f38fcaSMauro Carvalho Chehab ``V4L2_SEL_TGT_CROP`` 46354f38fcaSMauro Carvalho Chehab the rectangle within the coded resolution to be output to 46454f38fcaSMauro Carvalho Chehab ``CAPTURE``; defaults to ``V4L2_SEL_TGT_CROP_DEFAULT``; read-only on 46554f38fcaSMauro Carvalho Chehab hardware without additional compose/scaling capabilities. 46654f38fcaSMauro Carvalho Chehab 46754f38fcaSMauro Carvalho Chehab ``V4L2_SEL_TGT_COMPOSE_BOUNDS`` 46854f38fcaSMauro Carvalho Chehab the maximum rectangle within a ``CAPTURE`` buffer, which the cropped 46954f38fcaSMauro Carvalho Chehab frame can be composed into; equal to ``V4L2_SEL_TGT_CROP`` if the 47054f38fcaSMauro Carvalho Chehab hardware does not support compose/scaling. 47154f38fcaSMauro Carvalho Chehab 47254f38fcaSMauro Carvalho Chehab ``V4L2_SEL_TGT_COMPOSE_DEFAULT`` 47354f38fcaSMauro Carvalho Chehab equal to ``V4L2_SEL_TGT_CROP``. 47454f38fcaSMauro Carvalho Chehab 47554f38fcaSMauro Carvalho Chehab ``V4L2_SEL_TGT_COMPOSE`` 47654f38fcaSMauro Carvalho Chehab the rectangle inside a ``CAPTURE`` buffer into which the cropped 47754f38fcaSMauro Carvalho Chehab frame is written; defaults to ``V4L2_SEL_TGT_COMPOSE_DEFAULT``; 47854f38fcaSMauro Carvalho Chehab read-only on hardware without additional compose/scaling capabilities. 47954f38fcaSMauro Carvalho Chehab 48054f38fcaSMauro Carvalho Chehab ``V4L2_SEL_TGT_COMPOSE_PADDED`` 48154f38fcaSMauro Carvalho Chehab the rectangle inside a ``CAPTURE`` buffer which is overwritten by the 48254f38fcaSMauro Carvalho Chehab hardware; equal to ``V4L2_SEL_TGT_COMPOSE`` if the hardware does not 48354f38fcaSMauro Carvalho Chehab write padding pixels. 48454f38fcaSMauro Carvalho Chehab 48554f38fcaSMauro Carvalho Chehab .. warning:: 48654f38fcaSMauro Carvalho Chehab 48754f38fcaSMauro Carvalho Chehab The values are guaranteed to be meaningful only after the decoder 48854f38fcaSMauro Carvalho Chehab successfully parses the stream metadata. The client must not rely on the 48954f38fcaSMauro Carvalho Chehab query before that happens. 49054f38fcaSMauro Carvalho Chehab 49154f38fcaSMauro Carvalho Chehab3. **Optional.** Enumerate ``CAPTURE`` formats via :c:func:`VIDIOC_ENUM_FMT` on 49254f38fcaSMauro Carvalho Chehab the ``CAPTURE`` queue. Once the stream information is parsed and known, the 49354f38fcaSMauro Carvalho Chehab client may use this ioctl to discover which raw formats are supported for 49454f38fcaSMauro Carvalho Chehab given stream and select one of them via :c:func:`VIDIOC_S_FMT`. 49554f38fcaSMauro Carvalho Chehab 49654f38fcaSMauro Carvalho Chehab .. important:: 49754f38fcaSMauro Carvalho Chehab 49854f38fcaSMauro Carvalho Chehab The decoder will return only formats supported for the currently 49954f38fcaSMauro Carvalho Chehab established coded format, as per the ``OUTPUT`` format and/or stream 50054f38fcaSMauro Carvalho Chehab metadata parsed in this initialization sequence, even if more formats 50154f38fcaSMauro Carvalho Chehab may be supported by the decoder in general. In other words, the set 50254f38fcaSMauro Carvalho Chehab returned will be a subset of the initial query mentioned in the 50354f38fcaSMauro Carvalho Chehab `Querying Capabilities` section. 50454f38fcaSMauro Carvalho Chehab 50554f38fcaSMauro Carvalho Chehab For example, a decoder may support YUV and RGB formats for resolutions 50654f38fcaSMauro Carvalho Chehab 1920x1088 and lower, but only YUV for higher resolutions (due to 50754f38fcaSMauro Carvalho Chehab hardware limitations). After parsing a resolution of 1920x1088 or lower, 50854f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_ENUM_FMT` may return a set of YUV and RGB pixel formats, 50954f38fcaSMauro Carvalho Chehab but after parsing resolution higher than 1920x1088, the decoder will not 51054f38fcaSMauro Carvalho Chehab return RGB, unsupported for this resolution. 51154f38fcaSMauro Carvalho Chehab 51254f38fcaSMauro Carvalho Chehab However, subsequent resolution change event triggered after 51354f38fcaSMauro Carvalho Chehab discovering a resolution change within the same stream may switch 51454f38fcaSMauro Carvalho Chehab the stream into a lower resolution and :c:func:`VIDIOC_ENUM_FMT` 51554f38fcaSMauro Carvalho Chehab would return RGB formats again in that case. 51654f38fcaSMauro Carvalho Chehab 51754f38fcaSMauro Carvalho Chehab4. **Optional.** Set the ``CAPTURE`` format via :c:func:`VIDIOC_S_FMT` on the 51854f38fcaSMauro Carvalho Chehab ``CAPTURE`` queue. The client may choose a different format than 51954f38fcaSMauro Carvalho Chehab selected/suggested by the decoder in :c:func:`VIDIOC_G_FMT`. 52054f38fcaSMauro Carvalho Chehab 52154f38fcaSMauro Carvalho Chehab * **Required fields:** 52254f38fcaSMauro Carvalho Chehab 52354f38fcaSMauro Carvalho Chehab ``type`` 52454f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 52554f38fcaSMauro Carvalho Chehab 52654f38fcaSMauro Carvalho Chehab ``pixelformat`` 52754f38fcaSMauro Carvalho Chehab a raw pixel format. 52854f38fcaSMauro Carvalho Chehab 52954f38fcaSMauro Carvalho Chehab ``width``, ``height`` 53054f38fcaSMauro Carvalho Chehab frame buffer resolution of the decoded stream; typically unchanged from 53154f38fcaSMauro Carvalho Chehab what was returned with :c:func:`VIDIOC_G_FMT`, but it may be different 53254f38fcaSMauro Carvalho Chehab if the hardware supports composition and/or scaling. 53354f38fcaSMauro Carvalho Chehab 53454f38fcaSMauro Carvalho Chehab * Setting the ``CAPTURE`` format will reset the compose selection rectangles 53554f38fcaSMauro Carvalho Chehab to their default values, based on the new resolution, as described in the 53654f38fcaSMauro Carvalho Chehab previous step. 53754f38fcaSMauro Carvalho Chehab 53854f38fcaSMauro Carvalho Chehab5. **Optional.** Set the compose rectangle via :c:func:`VIDIOC_S_SELECTION` on 53954f38fcaSMauro Carvalho Chehab the ``CAPTURE`` queue if it is desired and if the decoder has compose and/or 54054f38fcaSMauro Carvalho Chehab scaling capabilities. 54154f38fcaSMauro Carvalho Chehab 54254f38fcaSMauro Carvalho Chehab * **Required fields:** 54354f38fcaSMauro Carvalho Chehab 54454f38fcaSMauro Carvalho Chehab ``type`` 54554f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 54654f38fcaSMauro Carvalho Chehab 54754f38fcaSMauro Carvalho Chehab ``target`` 54854f38fcaSMauro Carvalho Chehab set to ``V4L2_SEL_TGT_COMPOSE``. 54954f38fcaSMauro Carvalho Chehab 55054f38fcaSMauro Carvalho Chehab ``r.left``, ``r.top``, ``r.width``, ``r.height`` 55154f38fcaSMauro Carvalho Chehab the rectangle inside a ``CAPTURE`` buffer into which the cropped 55254f38fcaSMauro Carvalho Chehab frame is written; defaults to ``V4L2_SEL_TGT_COMPOSE_DEFAULT``; 55354f38fcaSMauro Carvalho Chehab read-only on hardware without additional compose/scaling capabilities. 55454f38fcaSMauro Carvalho Chehab 555*1073f441SPaul Kocialkowski * **Returned fields:** 55654f38fcaSMauro Carvalho Chehab 55754f38fcaSMauro Carvalho Chehab ``r.left``, ``r.top``, ``r.width``, ``r.height`` 55854f38fcaSMauro Carvalho Chehab the visible rectangle; it must fit within the frame buffer resolution 55954f38fcaSMauro Carvalho Chehab returned by :c:func:`VIDIOC_G_FMT` on ``CAPTURE``. 56054f38fcaSMauro Carvalho Chehab 56154f38fcaSMauro Carvalho Chehab .. warning:: 56254f38fcaSMauro Carvalho Chehab 56354f38fcaSMauro Carvalho Chehab The decoder may adjust the compose rectangle to the nearest 56454f38fcaSMauro Carvalho Chehab supported one to meet codec and hardware requirements. The client needs 56554f38fcaSMauro Carvalho Chehab to check the adjusted rectangle returned by :c:func:`VIDIOC_S_SELECTION`. 56654f38fcaSMauro Carvalho Chehab 56754f38fcaSMauro Carvalho Chehab6. If all the following conditions are met, the client may resume the decoding 56854f38fcaSMauro Carvalho Chehab instantly: 56954f38fcaSMauro Carvalho Chehab 57054f38fcaSMauro Carvalho Chehab * ``sizeimage`` of the new format (determined in previous steps) is less 57154f38fcaSMauro Carvalho Chehab than or equal to the size of currently allocated buffers, 57254f38fcaSMauro Carvalho Chehab 57354f38fcaSMauro Carvalho Chehab * the number of buffers currently allocated is greater than or equal to the 57454f38fcaSMauro Carvalho Chehab minimum number of buffers acquired in previous steps. To fulfill this 57554f38fcaSMauro Carvalho Chehab requirement, the client may use :c:func:`VIDIOC_CREATE_BUFS` to add new 57654f38fcaSMauro Carvalho Chehab buffers. 57754f38fcaSMauro Carvalho Chehab 57854f38fcaSMauro Carvalho Chehab In that case, the remaining steps do not apply and the client may resume 57954f38fcaSMauro Carvalho Chehab the decoding by one of the following actions: 58054f38fcaSMauro Carvalho Chehab 58154f38fcaSMauro Carvalho Chehab * if the ``CAPTURE`` queue is streaming, call :c:func:`VIDIOC_DECODER_CMD` 58254f38fcaSMauro Carvalho Chehab with the ``V4L2_DEC_CMD_START`` command, 58354f38fcaSMauro Carvalho Chehab 58454f38fcaSMauro Carvalho Chehab * if the ``CAPTURE`` queue is not streaming, call :c:func:`VIDIOC_STREAMON` 58554f38fcaSMauro Carvalho Chehab on the ``CAPTURE`` queue. 58654f38fcaSMauro Carvalho Chehab 58754f38fcaSMauro Carvalho Chehab However, if the client intends to change the buffer set, to lower 58854f38fcaSMauro Carvalho Chehab memory usage or for any other reasons, it may be achieved by following 58954f38fcaSMauro Carvalho Chehab the steps below. 59054f38fcaSMauro Carvalho Chehab 59154f38fcaSMauro Carvalho Chehab7. **If the** ``CAPTURE`` **queue is streaming,** keep queuing and dequeuing 59254f38fcaSMauro Carvalho Chehab buffers on the ``CAPTURE`` queue until a buffer marked with the 59354f38fcaSMauro Carvalho Chehab ``V4L2_BUF_FLAG_LAST`` flag is dequeued. 59454f38fcaSMauro Carvalho Chehab 59554f38fcaSMauro Carvalho Chehab8. **If the** ``CAPTURE`` **queue is streaming,** call :c:func:`VIDIOC_STREAMOFF` 59654f38fcaSMauro Carvalho Chehab on the ``CAPTURE`` queue to stop streaming. 59754f38fcaSMauro Carvalho Chehab 59854f38fcaSMauro Carvalho Chehab .. warning:: 59954f38fcaSMauro Carvalho Chehab 60054f38fcaSMauro Carvalho Chehab The ``OUTPUT`` queue must remain streaming. Calling 60154f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_STREAMOFF` on it would abort the sequence and trigger a 60254f38fcaSMauro Carvalho Chehab seek. 60354f38fcaSMauro Carvalho Chehab 60454f38fcaSMauro Carvalho Chehab9. **If the** ``CAPTURE`` **queue has buffers allocated,** free the ``CAPTURE`` 60554f38fcaSMauro Carvalho Chehab buffers using :c:func:`VIDIOC_REQBUFS`. 60654f38fcaSMauro Carvalho Chehab 60754f38fcaSMauro Carvalho Chehab * **Required fields:** 60854f38fcaSMauro Carvalho Chehab 60954f38fcaSMauro Carvalho Chehab ``count`` 61054f38fcaSMauro Carvalho Chehab set to 0. 61154f38fcaSMauro Carvalho Chehab 61254f38fcaSMauro Carvalho Chehab ``type`` 61354f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 61454f38fcaSMauro Carvalho Chehab 61554f38fcaSMauro Carvalho Chehab ``memory`` 61654f38fcaSMauro Carvalho Chehab follows standard semantics. 61754f38fcaSMauro Carvalho Chehab 61854f38fcaSMauro Carvalho Chehab10. Allocate ``CAPTURE`` buffers via :c:func:`VIDIOC_REQBUFS` on the 61954f38fcaSMauro Carvalho Chehab ``CAPTURE`` queue. 62054f38fcaSMauro Carvalho Chehab 62154f38fcaSMauro Carvalho Chehab * **Required fields:** 62254f38fcaSMauro Carvalho Chehab 62354f38fcaSMauro Carvalho Chehab ``count`` 62454f38fcaSMauro Carvalho Chehab requested number of buffers to allocate; greater than zero. 62554f38fcaSMauro Carvalho Chehab 62654f38fcaSMauro Carvalho Chehab ``type`` 62754f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 62854f38fcaSMauro Carvalho Chehab 62954f38fcaSMauro Carvalho Chehab ``memory`` 63054f38fcaSMauro Carvalho Chehab follows standard semantics. 63154f38fcaSMauro Carvalho Chehab 632*1073f441SPaul Kocialkowski * **Returned fields:** 63354f38fcaSMauro Carvalho Chehab 63454f38fcaSMauro Carvalho Chehab ``count`` 63554f38fcaSMauro Carvalho Chehab actual number of buffers allocated. 63654f38fcaSMauro Carvalho Chehab 63754f38fcaSMauro Carvalho Chehab .. warning:: 63854f38fcaSMauro Carvalho Chehab 63954f38fcaSMauro Carvalho Chehab The actual number of allocated buffers may differ from the ``count`` 64054f38fcaSMauro Carvalho Chehab given. The client must check the updated value of ``count`` after the 64154f38fcaSMauro Carvalho Chehab call returns. 64254f38fcaSMauro Carvalho Chehab 64354f38fcaSMauro Carvalho Chehab .. note:: 64454f38fcaSMauro Carvalho Chehab 64554f38fcaSMauro Carvalho Chehab To allocate more than the minimum number of buffers (for pipeline 64654f38fcaSMauro Carvalho Chehab depth), the client may query the ``V4L2_CID_MIN_BUFFERS_FOR_CAPTURE`` 64754f38fcaSMauro Carvalho Chehab control to get the minimum number of buffers required, and pass the 64854f38fcaSMauro Carvalho Chehab obtained value plus the number of additional buffers needed in the 64954f38fcaSMauro Carvalho Chehab ``count`` field to :c:func:`VIDIOC_REQBUFS`. 65054f38fcaSMauro Carvalho Chehab 65154f38fcaSMauro Carvalho Chehab Alternatively, :c:func:`VIDIOC_CREATE_BUFS` on the ``CAPTURE`` queue can be 65254f38fcaSMauro Carvalho Chehab used to have more control over buffer allocation. For example, by 65354f38fcaSMauro Carvalho Chehab allocating buffers larger than the current ``CAPTURE`` format, future 65454f38fcaSMauro Carvalho Chehab resolution changes can be accommodated. 65554f38fcaSMauro Carvalho Chehab 65654f38fcaSMauro Carvalho Chehab * **Required fields:** 65754f38fcaSMauro Carvalho Chehab 65854f38fcaSMauro Carvalho Chehab ``count`` 65954f38fcaSMauro Carvalho Chehab requested number of buffers to allocate; greater than zero. 66054f38fcaSMauro Carvalho Chehab 66154f38fcaSMauro Carvalho Chehab ``type`` 66254f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 66354f38fcaSMauro Carvalho Chehab 66454f38fcaSMauro Carvalho Chehab ``memory`` 66554f38fcaSMauro Carvalho Chehab follows standard semantics. 66654f38fcaSMauro Carvalho Chehab 66754f38fcaSMauro Carvalho Chehab ``format`` 66854f38fcaSMauro Carvalho Chehab a format representing the maximum framebuffer resolution to be 66954f38fcaSMauro Carvalho Chehab accommodated by newly allocated buffers. 67054f38fcaSMauro Carvalho Chehab 671*1073f441SPaul Kocialkowski * **Returned fields:** 67254f38fcaSMauro Carvalho Chehab 67354f38fcaSMauro Carvalho Chehab ``count`` 67454f38fcaSMauro Carvalho Chehab adjusted to the number of allocated buffers. 67554f38fcaSMauro Carvalho Chehab 67654f38fcaSMauro Carvalho Chehab .. warning:: 67754f38fcaSMauro Carvalho Chehab 67854f38fcaSMauro Carvalho Chehab The actual number of allocated buffers may differ from the ``count`` 67954f38fcaSMauro Carvalho Chehab given. The client must check the updated value of ``count`` after the 68054f38fcaSMauro Carvalho Chehab call returns. 68154f38fcaSMauro Carvalho Chehab 68254f38fcaSMauro Carvalho Chehab .. note:: 68354f38fcaSMauro Carvalho Chehab 68454f38fcaSMauro Carvalho Chehab To allocate buffers for a format different than parsed from the stream 68554f38fcaSMauro Carvalho Chehab metadata, the client must proceed as follows, before the metadata 68654f38fcaSMauro Carvalho Chehab parsing is initiated: 68754f38fcaSMauro Carvalho Chehab 68854f38fcaSMauro Carvalho Chehab * set width and height of the ``OUTPUT`` format to desired coded resolution to 68954f38fcaSMauro Carvalho Chehab let the decoder configure the ``CAPTURE`` format appropriately, 69054f38fcaSMauro Carvalho Chehab 69154f38fcaSMauro Carvalho Chehab * query the ``CAPTURE`` format using :c:func:`VIDIOC_G_FMT` and save it 69254f38fcaSMauro Carvalho Chehab until this step. 69354f38fcaSMauro Carvalho Chehab 69454f38fcaSMauro Carvalho Chehab The format obtained in the query may be then used with 69554f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_CREATE_BUFS` in this step to allocate the buffers. 69654f38fcaSMauro Carvalho Chehab 69754f38fcaSMauro Carvalho Chehab11. Call :c:func:`VIDIOC_STREAMON` on the ``CAPTURE`` queue to start decoding 69854f38fcaSMauro Carvalho Chehab frames. 69954f38fcaSMauro Carvalho Chehab 70054f38fcaSMauro Carvalho ChehabDecoding 70154f38fcaSMauro Carvalho Chehab======== 70254f38fcaSMauro Carvalho Chehab 70354f38fcaSMauro Carvalho ChehabThis state is reached after the `Capture Setup` sequence finishes successfully. 70454f38fcaSMauro Carvalho ChehabIn this state, the client queues and dequeues buffers to both queues via 70554f38fcaSMauro Carvalho Chehab:c:func:`VIDIOC_QBUF` and :c:func:`VIDIOC_DQBUF`, following the standard 70654f38fcaSMauro Carvalho Chehabsemantics. 70754f38fcaSMauro Carvalho Chehab 70854f38fcaSMauro Carvalho ChehabThe content of the source ``OUTPUT`` buffers depends on the active coded pixel 70954f38fcaSMauro Carvalho Chehabformat and may be affected by codec-specific extended controls, as stated in 71054f38fcaSMauro Carvalho Chehabthe documentation of each format. 71154f38fcaSMauro Carvalho Chehab 71254f38fcaSMauro Carvalho ChehabBoth queues operate independently, following the standard behavior of V4L2 71354f38fcaSMauro Carvalho Chehabbuffer queues and memory-to-memory devices. In addition, the order of decoded 71454f38fcaSMauro Carvalho Chehabframes dequeued from the ``CAPTURE`` queue may differ from the order of queuing 71554f38fcaSMauro Carvalho Chehabcoded frames to the ``OUTPUT`` queue, due to properties of the selected coded 71654f38fcaSMauro Carvalho Chehabformat, e.g. frame reordering. 71754f38fcaSMauro Carvalho Chehab 71854f38fcaSMauro Carvalho ChehabThe client must not assume any direct relationship between ``CAPTURE`` 71954f38fcaSMauro Carvalho Chehaband ``OUTPUT`` buffers and any specific timing of buffers becoming 72054f38fcaSMauro Carvalho Chehabavailable to dequeue. Specifically: 72154f38fcaSMauro Carvalho Chehab 72254f38fcaSMauro Carvalho Chehab* a buffer queued to ``OUTPUT`` may result in no buffers being produced 72354f38fcaSMauro Carvalho Chehab on ``CAPTURE`` (e.g. if it does not contain encoded data, or if only 72454f38fcaSMauro Carvalho Chehab metadata syntax structures are present in it), 72554f38fcaSMauro Carvalho Chehab 72654f38fcaSMauro Carvalho Chehab* a buffer queued to ``OUTPUT`` may result in more than one buffer produced 72754f38fcaSMauro Carvalho Chehab on ``CAPTURE`` (if the encoded data contained more than one frame, or if 72854f38fcaSMauro Carvalho Chehab returning a decoded frame allowed the decoder to return a frame that 72954f38fcaSMauro Carvalho Chehab preceded it in decode, but succeeded it in the display order), 73054f38fcaSMauro Carvalho Chehab 73154f38fcaSMauro Carvalho Chehab* a buffer queued to ``OUTPUT`` may result in a buffer being produced on 73254f38fcaSMauro Carvalho Chehab ``CAPTURE`` later into decode process, and/or after processing further 73354f38fcaSMauro Carvalho Chehab ``OUTPUT`` buffers, or be returned out of order, e.g. if display 73454f38fcaSMauro Carvalho Chehab reordering is used, 73554f38fcaSMauro Carvalho Chehab 73654f38fcaSMauro Carvalho Chehab* buffers may become available on the ``CAPTURE`` queue without additional 73754f38fcaSMauro Carvalho Chehab buffers queued to ``OUTPUT`` (e.g. during drain or ``EOS``), because of the 73854f38fcaSMauro Carvalho Chehab ``OUTPUT`` buffers queued in the past whose decoding results are only 73954f38fcaSMauro Carvalho Chehab available at later time, due to specifics of the decoding process. 74054f38fcaSMauro Carvalho Chehab 74154f38fcaSMauro Carvalho Chehab.. note:: 74254f38fcaSMauro Carvalho Chehab 74354f38fcaSMauro Carvalho Chehab To allow matching decoded ``CAPTURE`` buffers with ``OUTPUT`` buffers they 74454f38fcaSMauro Carvalho Chehab originated from, the client can set the ``timestamp`` field of the 74554f38fcaSMauro Carvalho Chehab :c:type:`v4l2_buffer` struct when queuing an ``OUTPUT`` buffer. The 74654f38fcaSMauro Carvalho Chehab ``CAPTURE`` buffer(s), which resulted from decoding that ``OUTPUT`` buffer 74754f38fcaSMauro Carvalho Chehab will have their ``timestamp`` field set to the same value when dequeued. 74854f38fcaSMauro Carvalho Chehab 74954f38fcaSMauro Carvalho Chehab In addition to the straightforward case of one ``OUTPUT`` buffer producing 75054f38fcaSMauro Carvalho Chehab one ``CAPTURE`` buffer, the following cases are defined: 75154f38fcaSMauro Carvalho Chehab 75254f38fcaSMauro Carvalho Chehab * one ``OUTPUT`` buffer generates multiple ``CAPTURE`` buffers: the same 75354f38fcaSMauro Carvalho Chehab ``OUTPUT`` timestamp will be copied to multiple ``CAPTURE`` buffers. 75454f38fcaSMauro Carvalho Chehab 75554f38fcaSMauro Carvalho Chehab * multiple ``OUTPUT`` buffers generate one ``CAPTURE`` buffer: timestamp of 75654f38fcaSMauro Carvalho Chehab the ``OUTPUT`` buffer queued first will be copied. 75754f38fcaSMauro Carvalho Chehab 75854f38fcaSMauro Carvalho Chehab * the decoding order differs from the display order (i.e. the ``CAPTURE`` 75954f38fcaSMauro Carvalho Chehab buffers are out-of-order compared to the ``OUTPUT`` buffers): ``CAPTURE`` 76054f38fcaSMauro Carvalho Chehab timestamps will not retain the order of ``OUTPUT`` timestamps. 76154f38fcaSMauro Carvalho Chehab 7624cfe98e6SAlexandre Courbot.. note:: 7634cfe98e6SAlexandre Courbot 7644cfe98e6SAlexandre Courbot The backing memory of ``CAPTURE`` buffers that are used as reference frames 7654cfe98e6SAlexandre Courbot by the stream may be read by the hardware even after they are dequeued. 7664cfe98e6SAlexandre Courbot Consequently, the client should avoid writing into this memory while the 7674cfe98e6SAlexandre Courbot ``CAPTURE`` queue is streaming. Failure to observe this may result in 7684cfe98e6SAlexandre Courbot corruption of decoded frames. 7694cfe98e6SAlexandre Courbot 7704cfe98e6SAlexandre Courbot Similarly, when using a memory type other than ``V4L2_MEMORY_MMAP``, the 7714cfe98e6SAlexandre Courbot client should make sure that each ``CAPTURE`` buffer is always queued with 7724cfe98e6SAlexandre Courbot the same backing memory for as long as the ``CAPTURE`` queue is streaming. 7734cfe98e6SAlexandre Courbot The reason for this is that V4L2 buffer indices can be used by drivers to 7744cfe98e6SAlexandre Courbot identify frames. Thus, if the backing memory of a reference frame is 7754cfe98e6SAlexandre Courbot submitted under a different buffer ID, the driver may misidentify it and 7764cfe98e6SAlexandre Courbot decode a new frame into it while it is still in use, resulting in corruption 7774cfe98e6SAlexandre Courbot of the following frames. 7784cfe98e6SAlexandre Courbot 77954f38fcaSMauro Carvalho ChehabDuring the decoding, the decoder may initiate one of the special sequences, as 78054f38fcaSMauro Carvalho Chehablisted below. The sequences will result in the decoder returning all the 78154f38fcaSMauro Carvalho Chehab``CAPTURE`` buffers that originated from all the ``OUTPUT`` buffers processed 78254f38fcaSMauro Carvalho Chehabbefore the sequence started. Last of the buffers will have the 78354f38fcaSMauro Carvalho Chehab``V4L2_BUF_FLAG_LAST`` flag set. To determine the sequence to follow, the client 78454f38fcaSMauro Carvalho Chehabmust check if there is any pending event and: 78554f38fcaSMauro Carvalho Chehab 78654f38fcaSMauro Carvalho Chehab* if a ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to 78754f38fcaSMauro Carvalho Chehab ``V4L2_EVENT_SRC_CH_RESOLUTION`` is pending, the `Dynamic Resolution 78854f38fcaSMauro Carvalho Chehab Change` sequence needs to be followed, 78954f38fcaSMauro Carvalho Chehab 79054f38fcaSMauro Carvalho Chehab* if a ``V4L2_EVENT_EOS`` event is pending, the `End of Stream` sequence needs 79154f38fcaSMauro Carvalho Chehab to be followed. 79254f38fcaSMauro Carvalho Chehab 79354f38fcaSMauro Carvalho ChehabSome of the sequences can be intermixed with each other and need to be handled 79454f38fcaSMauro Carvalho Chehabas they happen. The exact operation is documented for each sequence. 79554f38fcaSMauro Carvalho Chehab 79654f38fcaSMauro Carvalho ChehabShould a decoding error occur, it will be reported to the client with the level 79754f38fcaSMauro Carvalho Chehabof details depending on the decoder capabilities. Specifically: 79854f38fcaSMauro Carvalho Chehab 79954f38fcaSMauro Carvalho Chehab* the CAPTURE buffer that contains the results of the failed decode operation 80054f38fcaSMauro Carvalho Chehab will be returned with the V4L2_BUF_FLAG_ERROR flag set, 80154f38fcaSMauro Carvalho Chehab 80254f38fcaSMauro Carvalho Chehab* if the decoder is able to precisely report the OUTPUT buffer that triggered 80354f38fcaSMauro Carvalho Chehab the error, such buffer will be returned with the V4L2_BUF_FLAG_ERROR flag 80454f38fcaSMauro Carvalho Chehab set. 80554f38fcaSMauro Carvalho Chehab 80654f38fcaSMauro Carvalho ChehabIn case of a fatal failure that does not allow the decoding to continue, any 80754f38fcaSMauro Carvalho Chehabfurther operations on corresponding decoder file handle will return the -EIO 80854f38fcaSMauro Carvalho Chehaberror code. The client may close the file handle and open a new one, or 80954f38fcaSMauro Carvalho Chehabalternatively reinitialize the instance by stopping streaming on both queues, 81054f38fcaSMauro Carvalho Chehabreleasing all buffers and performing the Initialization sequence again. 81154f38fcaSMauro Carvalho Chehab 81254f38fcaSMauro Carvalho ChehabSeek 81354f38fcaSMauro Carvalho Chehab==== 81454f38fcaSMauro Carvalho Chehab 81554f38fcaSMauro Carvalho ChehabSeek is controlled by the ``OUTPUT`` queue, as it is the source of coded data. 81654f38fcaSMauro Carvalho ChehabThe seek does not require any specific operation on the ``CAPTURE`` queue, but 81754f38fcaSMauro Carvalho Chehabit may be affected as per normal decoder operation. 81854f38fcaSMauro Carvalho Chehab 81954f38fcaSMauro Carvalho Chehab1. Stop the ``OUTPUT`` queue to begin the seek sequence via 82054f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_STREAMOFF`. 82154f38fcaSMauro Carvalho Chehab 82254f38fcaSMauro Carvalho Chehab * **Required fields:** 82354f38fcaSMauro Carvalho Chehab 82454f38fcaSMauro Carvalho Chehab ``type`` 82554f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 82654f38fcaSMauro Carvalho Chehab 82754f38fcaSMauro Carvalho Chehab * The decoder will drop all the pending ``OUTPUT`` buffers and they must be 82854f38fcaSMauro Carvalho Chehab treated as returned to the client (following standard semantics). 82954f38fcaSMauro Carvalho Chehab 830eef7a5e7SHans Verkuil2. Restart the ``OUTPUT`` queue via :c:func:`VIDIOC_STREAMON`. 83154f38fcaSMauro Carvalho Chehab 83254f38fcaSMauro Carvalho Chehab * **Required fields:** 83354f38fcaSMauro Carvalho Chehab 83454f38fcaSMauro Carvalho Chehab ``type`` 83554f38fcaSMauro Carvalho Chehab a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 83654f38fcaSMauro Carvalho Chehab 83754f38fcaSMauro Carvalho Chehab * The decoder will start accepting new source bytestream buffers after the 83854f38fcaSMauro Carvalho Chehab call returns. 83954f38fcaSMauro Carvalho Chehab 84054f38fcaSMauro Carvalho Chehab3. Start queuing buffers containing coded data after the seek to the ``OUTPUT`` 84154f38fcaSMauro Carvalho Chehab queue until a suitable resume point is found. 84254f38fcaSMauro Carvalho Chehab 84354f38fcaSMauro Carvalho Chehab .. note:: 84454f38fcaSMauro Carvalho Chehab 84554f38fcaSMauro Carvalho Chehab There is no requirement to begin queuing coded data starting exactly 84654f38fcaSMauro Carvalho Chehab from a resume point (e.g. SPS or a keyframe). Any queued ``OUTPUT`` 84754f38fcaSMauro Carvalho Chehab buffers will be processed and returned to the client until a suitable 84854f38fcaSMauro Carvalho Chehab resume point is found. While looking for a resume point, the decoder 84954f38fcaSMauro Carvalho Chehab should not produce any decoded frames into ``CAPTURE`` buffers. 85054f38fcaSMauro Carvalho Chehab 85154f38fcaSMauro Carvalho Chehab Some hardware is known to mishandle seeks to a non-resume point. Such an 85254f38fcaSMauro Carvalho Chehab operation may result in an unspecified number of corrupted decoded frames 85354f38fcaSMauro Carvalho Chehab being made available on the ``CAPTURE`` queue. Drivers must ensure that 85454f38fcaSMauro Carvalho Chehab no fatal decoding errors or crashes occur, and implement any necessary 85554f38fcaSMauro Carvalho Chehab handling and workarounds for hardware issues related to seek operations. 85654f38fcaSMauro Carvalho Chehab 85754f38fcaSMauro Carvalho Chehab .. warning:: 85854f38fcaSMauro Carvalho Chehab 85954f38fcaSMauro Carvalho Chehab In case of the H.264/HEVC codec, the client must take care not to seek 86054f38fcaSMauro Carvalho Chehab over a change of SPS/PPS. Even though the target frame could be a 86154f38fcaSMauro Carvalho Chehab keyframe, the stale SPS/PPS inside decoder state would lead to undefined 86254f38fcaSMauro Carvalho Chehab results when decoding. Although the decoder must handle that case without 86354f38fcaSMauro Carvalho Chehab a crash or a fatal decode error, the client must not expect a sensible 86454f38fcaSMauro Carvalho Chehab decode output. 86554f38fcaSMauro Carvalho Chehab 86654f38fcaSMauro Carvalho Chehab If the hardware can detect such corrupted decoded frames, then 86754f38fcaSMauro Carvalho Chehab corresponding buffers will be returned to the client with the 86854f38fcaSMauro Carvalho Chehab V4L2_BUF_FLAG_ERROR set. See the `Decoding` section for further 86954f38fcaSMauro Carvalho Chehab description of decode error reporting. 87054f38fcaSMauro Carvalho Chehab 87154f38fcaSMauro Carvalho Chehab4. After a resume point is found, the decoder will start returning ``CAPTURE`` 87254f38fcaSMauro Carvalho Chehab buffers containing decoded frames. 87354f38fcaSMauro Carvalho Chehab 87454f38fcaSMauro Carvalho Chehab.. important:: 87554f38fcaSMauro Carvalho Chehab 87654f38fcaSMauro Carvalho Chehab A seek may result in the `Dynamic Resolution Change` sequence being 87754f38fcaSMauro Carvalho Chehab initiated, due to the seek target having decoding parameters different from 87854f38fcaSMauro Carvalho Chehab the part of the stream decoded before the seek. The sequence must be handled 87954f38fcaSMauro Carvalho Chehab as per normal decoder operation. 88054f38fcaSMauro Carvalho Chehab 88154f38fcaSMauro Carvalho Chehab.. warning:: 88254f38fcaSMauro Carvalho Chehab 88354f38fcaSMauro Carvalho Chehab It is not specified when the ``CAPTURE`` queue starts producing buffers 88454f38fcaSMauro Carvalho Chehab containing decoded data from the ``OUTPUT`` buffers queued after the seek, 88554f38fcaSMauro Carvalho Chehab as it operates independently from the ``OUTPUT`` queue. 88654f38fcaSMauro Carvalho Chehab 88754f38fcaSMauro Carvalho Chehab The decoder may return a number of remaining ``CAPTURE`` buffers containing 88854f38fcaSMauro Carvalho Chehab decoded frames originating from the ``OUTPUT`` buffers queued before the 88954f38fcaSMauro Carvalho Chehab seek sequence is performed. 89054f38fcaSMauro Carvalho Chehab 89154f38fcaSMauro Carvalho Chehab The ``VIDIOC_STREAMOFF`` operation discards any remaining queued 89254f38fcaSMauro Carvalho Chehab ``OUTPUT`` buffers, which means that not all of the ``OUTPUT`` buffers 89354f38fcaSMauro Carvalho Chehab queued before the seek sequence may have matching ``CAPTURE`` buffers 89454f38fcaSMauro Carvalho Chehab produced. For example, given the sequence of operations on the 89554f38fcaSMauro Carvalho Chehab ``OUTPUT`` queue: 89654f38fcaSMauro Carvalho Chehab 89754f38fcaSMauro Carvalho Chehab QBUF(A), QBUF(B), STREAMOFF(), STREAMON(), QBUF(G), QBUF(H), 89854f38fcaSMauro Carvalho Chehab 89954f38fcaSMauro Carvalho Chehab any of the following results on the ``CAPTURE`` queue is allowed: 90054f38fcaSMauro Carvalho Chehab 901eff7d26aSMauro Carvalho Chehab {A', B', G', H'}, {A', G', H'}, {G', H'}. 90254f38fcaSMauro Carvalho Chehab 90354f38fcaSMauro Carvalho Chehab To determine the CAPTURE buffer containing the first decoded frame after the 90454f38fcaSMauro Carvalho Chehab seek, the client may observe the timestamps to match the CAPTURE and OUTPUT 90554f38fcaSMauro Carvalho Chehab buffers or use V4L2_DEC_CMD_STOP and V4L2_DEC_CMD_START to drain the 90654f38fcaSMauro Carvalho Chehab decoder. 90754f38fcaSMauro Carvalho Chehab 90854f38fcaSMauro Carvalho Chehab.. note:: 90954f38fcaSMauro Carvalho Chehab 91054f38fcaSMauro Carvalho Chehab To achieve instantaneous seek, the client may restart streaming on the 91154f38fcaSMauro Carvalho Chehab ``CAPTURE`` queue too to discard decoded, but not yet dequeued buffers. 91254f38fcaSMauro Carvalho Chehab 91354f38fcaSMauro Carvalho ChehabDynamic Resolution Change 91454f38fcaSMauro Carvalho Chehab========================= 91554f38fcaSMauro Carvalho Chehab 91654f38fcaSMauro Carvalho ChehabStreams that include resolution metadata in the bytestream may require switching 91754f38fcaSMauro Carvalho Chehabto a different resolution during the decoding. 91854f38fcaSMauro Carvalho Chehab 91954f38fcaSMauro Carvalho Chehab.. note:: 92054f38fcaSMauro Carvalho Chehab 92154f38fcaSMauro Carvalho Chehab Not all decoders can detect resolution changes. Those that do set the 92254f38fcaSMauro Carvalho Chehab ``V4L2_FMT_FLAG_DYN_RESOLUTION`` flag for the coded format when 92354f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_ENUM_FMT` is called. 92454f38fcaSMauro Carvalho Chehab 92554f38fcaSMauro Carvalho ChehabThe sequence starts when the decoder detects a coded frame with one or more of 92654f38fcaSMauro Carvalho Chehabthe following parameters different from those previously established (and 92754f38fcaSMauro Carvalho Chehabreflected by corresponding queries): 92854f38fcaSMauro Carvalho Chehab 92954f38fcaSMauro Carvalho Chehab* coded resolution (``OUTPUT`` width and height), 93054f38fcaSMauro Carvalho Chehab 93154f38fcaSMauro Carvalho Chehab* visible resolution (selection rectangles), 93254f38fcaSMauro Carvalho Chehab 933874b8055SStanimir Varbanov* the minimum number of buffers needed for decoding, 934874b8055SStanimir Varbanov 935874b8055SStanimir Varbanov* bit-depth of the bitstream has been changed. 93654f38fcaSMauro Carvalho Chehab 93754f38fcaSMauro Carvalho ChehabWhenever that happens, the decoder must proceed as follows: 93854f38fcaSMauro Carvalho Chehab 93954f38fcaSMauro Carvalho Chehab1. After encountering a resolution change in the stream, the decoder sends a 94054f38fcaSMauro Carvalho Chehab ``V4L2_EVENT_SOURCE_CHANGE`` event with ``changes`` set to 94154f38fcaSMauro Carvalho Chehab ``V4L2_EVENT_SRC_CH_RESOLUTION``. 94254f38fcaSMauro Carvalho Chehab 94354f38fcaSMauro Carvalho Chehab .. important:: 94454f38fcaSMauro Carvalho Chehab 94554f38fcaSMauro Carvalho Chehab Any client query issued after the decoder queues the event will return 94654f38fcaSMauro Carvalho Chehab values applying to the stream after the resolution change, including 94754f38fcaSMauro Carvalho Chehab queue formats, selection rectangles and controls. 94854f38fcaSMauro Carvalho Chehab 94954f38fcaSMauro Carvalho Chehab2. The decoder will then process and decode all remaining buffers from before 95054f38fcaSMauro Carvalho Chehab the resolution change point. 95154f38fcaSMauro Carvalho Chehab 95254f38fcaSMauro Carvalho Chehab * The last buffer from before the change must be marked with the 95354f38fcaSMauro Carvalho Chehab ``V4L2_BUF_FLAG_LAST`` flag, similarly to the `Drain` sequence above. 95454f38fcaSMauro Carvalho Chehab 95554f38fcaSMauro Carvalho Chehab .. warning:: 95654f38fcaSMauro Carvalho Chehab 95754f38fcaSMauro Carvalho Chehab The last buffer may be empty (with :c:type:`v4l2_buffer` ``bytesused`` 95854f38fcaSMauro Carvalho Chehab = 0) and in that case it must be ignored by the client, as it does not 95954f38fcaSMauro Carvalho Chehab contain a decoded frame. 96054f38fcaSMauro Carvalho Chehab 96154f38fcaSMauro Carvalho Chehab .. note:: 96254f38fcaSMauro Carvalho Chehab 96354f38fcaSMauro Carvalho Chehab Any attempt to dequeue more ``CAPTURE`` buffers beyond the buffer marked 96454f38fcaSMauro Carvalho Chehab with ``V4L2_BUF_FLAG_LAST`` will result in a -EPIPE error from 96554f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_DQBUF`. 96654f38fcaSMauro Carvalho Chehab 96754f38fcaSMauro Carvalho ChehabThe client must continue the sequence as described below to continue the 96854f38fcaSMauro Carvalho Chehabdecoding process. 96954f38fcaSMauro Carvalho Chehab 97054f38fcaSMauro Carvalho Chehab1. Dequeue the source change event. 97154f38fcaSMauro Carvalho Chehab 97254f38fcaSMauro Carvalho Chehab .. important:: 97354f38fcaSMauro Carvalho Chehab 97454f38fcaSMauro Carvalho Chehab A source change triggers an implicit decoder drain, similar to the 97554f38fcaSMauro Carvalho Chehab explicit `Drain` sequence. The decoder is stopped after it completes. 97654f38fcaSMauro Carvalho Chehab The decoding process must be resumed with either a pair of calls to 97754f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the 97854f38fcaSMauro Carvalho Chehab ``CAPTURE`` queue, or a call to :c:func:`VIDIOC_DECODER_CMD` with the 97954f38fcaSMauro Carvalho Chehab ``V4L2_DEC_CMD_START`` command. 98054f38fcaSMauro Carvalho Chehab 98154f38fcaSMauro Carvalho Chehab2. Continue with the `Capture Setup` sequence. 98254f38fcaSMauro Carvalho Chehab 98354f38fcaSMauro Carvalho Chehab.. note:: 98454f38fcaSMauro Carvalho Chehab 98554f38fcaSMauro Carvalho Chehab During the resolution change sequence, the ``OUTPUT`` queue must remain 98654f38fcaSMauro Carvalho Chehab streaming. Calling :c:func:`VIDIOC_STREAMOFF` on the ``OUTPUT`` queue would 98754f38fcaSMauro Carvalho Chehab abort the sequence and initiate a seek. 98854f38fcaSMauro Carvalho Chehab 98954f38fcaSMauro Carvalho Chehab In principle, the ``OUTPUT`` queue operates separately from the ``CAPTURE`` 99054f38fcaSMauro Carvalho Chehab queue and this remains true for the duration of the entire resolution change 99154f38fcaSMauro Carvalho Chehab sequence as well. 99254f38fcaSMauro Carvalho Chehab 99354f38fcaSMauro Carvalho Chehab The client should, for best performance and simplicity, keep queuing/dequeuing 99454f38fcaSMauro Carvalho Chehab buffers to/from the ``OUTPUT`` queue even while processing this sequence. 99554f38fcaSMauro Carvalho Chehab 99654f38fcaSMauro Carvalho ChehabDrain 99754f38fcaSMauro Carvalho Chehab===== 99854f38fcaSMauro Carvalho Chehab 99954f38fcaSMauro Carvalho ChehabTo ensure that all queued ``OUTPUT`` buffers have been processed and related 100054f38fcaSMauro Carvalho Chehab``CAPTURE`` buffers are given to the client, the client must follow the drain 100154f38fcaSMauro Carvalho Chehabsequence described below. After the drain sequence ends, the client has 100254f38fcaSMauro Carvalho Chehabreceived all decoded frames for all ``OUTPUT`` buffers queued before the 100354f38fcaSMauro Carvalho Chehabsequence was started. 100454f38fcaSMauro Carvalho Chehab 100554f38fcaSMauro Carvalho Chehab1. Begin drain by issuing :c:func:`VIDIOC_DECODER_CMD`. 100654f38fcaSMauro Carvalho Chehab 100754f38fcaSMauro Carvalho Chehab * **Required fields:** 100854f38fcaSMauro Carvalho Chehab 100954f38fcaSMauro Carvalho Chehab ``cmd`` 101054f38fcaSMauro Carvalho Chehab set to ``V4L2_DEC_CMD_STOP``. 101154f38fcaSMauro Carvalho Chehab 101254f38fcaSMauro Carvalho Chehab ``flags`` 101354f38fcaSMauro Carvalho Chehab set to 0. 101454f38fcaSMauro Carvalho Chehab 101554f38fcaSMauro Carvalho Chehab ``pts`` 101654f38fcaSMauro Carvalho Chehab set to 0. 101754f38fcaSMauro Carvalho Chehab 101854f38fcaSMauro Carvalho Chehab .. warning:: 101954f38fcaSMauro Carvalho Chehab 102054f38fcaSMauro Carvalho Chehab The sequence can be only initiated if both ``OUTPUT`` and ``CAPTURE`` 102154f38fcaSMauro Carvalho Chehab queues are streaming. For compatibility reasons, the call to 102254f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_DECODER_CMD` will not fail even if any of the queues is 102354f38fcaSMauro Carvalho Chehab not streaming, but at the same time it will not initiate the `Drain` 102454f38fcaSMauro Carvalho Chehab sequence and so the steps described below would not be applicable. 102554f38fcaSMauro Carvalho Chehab 102654f38fcaSMauro Carvalho Chehab2. Any ``OUTPUT`` buffers queued by the client before the 102754f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_DECODER_CMD` was issued will be processed and decoded as 102854f38fcaSMauro Carvalho Chehab normal. The client must continue to handle both queues independently, 102954f38fcaSMauro Carvalho Chehab similarly to normal decode operation. This includes: 103054f38fcaSMauro Carvalho Chehab 103154f38fcaSMauro Carvalho Chehab * handling any operations triggered as a result of processing those buffers, 103254f38fcaSMauro Carvalho Chehab such as the `Dynamic Resolution Change` sequence, before continuing with 103354f38fcaSMauro Carvalho Chehab the drain sequence, 103454f38fcaSMauro Carvalho Chehab 103554f38fcaSMauro Carvalho Chehab * queuing and dequeuing ``CAPTURE`` buffers, until a buffer marked with the 103654f38fcaSMauro Carvalho Chehab ``V4L2_BUF_FLAG_LAST`` flag is dequeued, 103754f38fcaSMauro Carvalho Chehab 103854f38fcaSMauro Carvalho Chehab .. warning:: 103954f38fcaSMauro Carvalho Chehab 104054f38fcaSMauro Carvalho Chehab The last buffer may be empty (with :c:type:`v4l2_buffer` 104154f38fcaSMauro Carvalho Chehab ``bytesused`` = 0) and in that case it must be ignored by the client, 104254f38fcaSMauro Carvalho Chehab as it does not contain a decoded frame. 104354f38fcaSMauro Carvalho Chehab 104454f38fcaSMauro Carvalho Chehab .. note:: 104554f38fcaSMauro Carvalho Chehab 104654f38fcaSMauro Carvalho Chehab Any attempt to dequeue more ``CAPTURE`` buffers beyond the buffer 104754f38fcaSMauro Carvalho Chehab marked with ``V4L2_BUF_FLAG_LAST`` will result in a -EPIPE error from 104854f38fcaSMauro Carvalho Chehab :c:func:`VIDIOC_DQBUF`. 104954f38fcaSMauro Carvalho Chehab 105054f38fcaSMauro Carvalho Chehab * dequeuing processed ``OUTPUT`` buffers, until all the buffers queued 105154f38fcaSMauro Carvalho Chehab before the ``V4L2_DEC_CMD_STOP`` command are dequeued, 105254f38fcaSMauro Carvalho Chehab 105354f38fcaSMauro Carvalho Chehab * dequeuing the ``V4L2_EVENT_EOS`` event, if the client subscribed to it. 105454f38fcaSMauro Carvalho Chehab 105554f38fcaSMauro Carvalho Chehab .. note:: 105654f38fcaSMauro Carvalho Chehab 105754f38fcaSMauro Carvalho Chehab For backwards compatibility, the decoder will signal a ``V4L2_EVENT_EOS`` 105854f38fcaSMauro Carvalho Chehab event when the last frame has been decoded and all frames are ready to be 105954f38fcaSMauro Carvalho Chehab dequeued. It is a deprecated behavior and the client must not rely on it. 106054f38fcaSMauro Carvalho Chehab The ``V4L2_BUF_FLAG_LAST`` buffer flag should be used instead. 106154f38fcaSMauro Carvalho Chehab 106254f38fcaSMauro Carvalho Chehab3. Once all the ``OUTPUT`` buffers queued before the ``V4L2_DEC_CMD_STOP`` call 106354f38fcaSMauro Carvalho Chehab are dequeued and the last ``CAPTURE`` buffer is dequeued, the decoder is 106454f38fcaSMauro Carvalho Chehab stopped and it will accept, but not process, any newly queued ``OUTPUT`` 106554f38fcaSMauro Carvalho Chehab buffers until the client issues any of the following operations: 106654f38fcaSMauro Carvalho Chehab 106754f38fcaSMauro Carvalho Chehab * ``V4L2_DEC_CMD_START`` - the decoder will not be reset and will resume 106854f38fcaSMauro Carvalho Chehab operation normally, with all the state from before the drain, 106954f38fcaSMauro Carvalho Chehab 107054f38fcaSMauro Carvalho Chehab * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the 107154f38fcaSMauro Carvalho Chehab ``CAPTURE`` queue - the decoder will resume the operation normally, 107254f38fcaSMauro Carvalho Chehab however any ``CAPTURE`` buffers still in the queue will be returned to the 107354f38fcaSMauro Carvalho Chehab client, 107454f38fcaSMauro Carvalho Chehab 107554f38fcaSMauro Carvalho Chehab * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the 107654f38fcaSMauro Carvalho Chehab ``OUTPUT`` queue - any pending source buffers will be returned to the 107754f38fcaSMauro Carvalho Chehab client and the `Seek` sequence will be triggered. 107854f38fcaSMauro Carvalho Chehab 107954f38fcaSMauro Carvalho Chehab.. note:: 108054f38fcaSMauro Carvalho Chehab 108154f38fcaSMauro Carvalho Chehab Once the drain sequence is initiated, the client needs to drive it to 108254f38fcaSMauro Carvalho Chehab completion, as described by the steps above, unless it aborts the process by 108354f38fcaSMauro Carvalho Chehab issuing :c:func:`VIDIOC_STREAMOFF` on any of the ``OUTPUT`` or ``CAPTURE`` 108454f38fcaSMauro Carvalho Chehab queues. The client is not allowed to issue ``V4L2_DEC_CMD_START`` or 108554f38fcaSMauro Carvalho Chehab ``V4L2_DEC_CMD_STOP`` again while the drain sequence is in progress and they 108654f38fcaSMauro Carvalho Chehab will fail with -EBUSY error code if attempted. 108754f38fcaSMauro Carvalho Chehab 1088eef7a5e7SHans Verkuil Although not mandatory, the availability of decoder commands may be queried 108954f38fcaSMauro Carvalho Chehab using :c:func:`VIDIOC_TRY_DECODER_CMD`. 109054f38fcaSMauro Carvalho Chehab 109154f38fcaSMauro Carvalho ChehabEnd of Stream 109254f38fcaSMauro Carvalho Chehab============= 109354f38fcaSMauro Carvalho Chehab 109454f38fcaSMauro Carvalho ChehabIf the decoder encounters an end of stream marking in the stream, the decoder 109554f38fcaSMauro Carvalho Chehabwill initiate the `Drain` sequence, which the client must handle as described 109654f38fcaSMauro Carvalho Chehababove, skipping the initial :c:func:`VIDIOC_DECODER_CMD`. 109754f38fcaSMauro Carvalho Chehab 109854f38fcaSMauro Carvalho ChehabCommit Points 109954f38fcaSMauro Carvalho Chehab============= 110054f38fcaSMauro Carvalho Chehab 110154f38fcaSMauro Carvalho ChehabSetting formats and allocating buffers trigger changes in the behavior of the 110254f38fcaSMauro Carvalho Chehabdecoder. 110354f38fcaSMauro Carvalho Chehab 110454f38fcaSMauro Carvalho Chehab1. Setting the format on the ``OUTPUT`` queue may change the set of formats 110554f38fcaSMauro Carvalho Chehab supported/advertised on the ``CAPTURE`` queue. In particular, it also means 110654f38fcaSMauro Carvalho Chehab that the ``CAPTURE`` format may be reset and the client must not rely on the 110754f38fcaSMauro Carvalho Chehab previously set format being preserved. 110854f38fcaSMauro Carvalho Chehab 110954f38fcaSMauro Carvalho Chehab2. Enumerating formats on the ``CAPTURE`` queue always returns only formats 111054f38fcaSMauro Carvalho Chehab supported for the current ``OUTPUT`` format. 111154f38fcaSMauro Carvalho Chehab 111254f38fcaSMauro Carvalho Chehab3. Setting the format on the ``CAPTURE`` queue does not change the list of 111354f38fcaSMauro Carvalho Chehab formats available on the ``OUTPUT`` queue. An attempt to set a ``CAPTURE`` 111454f38fcaSMauro Carvalho Chehab format that is not supported for the currently selected ``OUTPUT`` format 111554f38fcaSMauro Carvalho Chehab will result in the decoder adjusting the requested ``CAPTURE`` format to a 111654f38fcaSMauro Carvalho Chehab supported one. 111754f38fcaSMauro Carvalho Chehab 111854f38fcaSMauro Carvalho Chehab4. Enumerating formats on the ``OUTPUT`` queue always returns the full set of 111954f38fcaSMauro Carvalho Chehab supported coded formats, irrespectively of the current ``CAPTURE`` format. 112054f38fcaSMauro Carvalho Chehab 112154f38fcaSMauro Carvalho Chehab5. While buffers are allocated on any of the ``OUTPUT`` or ``CAPTURE`` queues, 112254f38fcaSMauro Carvalho Chehab the client must not change the format on the ``OUTPUT`` queue. Drivers will 112354f38fcaSMauro Carvalho Chehab return the -EBUSY error code for any such format change attempt. 112454f38fcaSMauro Carvalho Chehab 112554f38fcaSMauro Carvalho ChehabTo summarize, setting formats and allocation must always start with the 112654f38fcaSMauro Carvalho Chehab``OUTPUT`` queue and the ``OUTPUT`` queue is the master that governs the 112754f38fcaSMauro Carvalho Chehabset of supported formats for the ``CAPTURE`` queue. 1128