1.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later 2 3.. _overlay: 4 5*********************** 6Video Overlay Interface 7*********************** 8 9**Also known as Framebuffer Overlay or Previewing.** 10 11Video overlay devices have the ability to genlock (TV-)video into the 12(VGA-)video signal of a graphics card, or to store captured images 13directly in video memory of a graphics card, typically with clipping. 14This can be considerable more efficient than capturing images and 15displaying them by other means. In the old days when only nuclear power 16plants needed cooling towers this used to be the only way to put live 17video into a window. 18 19Video overlay devices are accessed through the same character special 20files as :ref:`video capture <capture>` devices. 21 22.. note:: 23 24 The default function of a ``/dev/video`` device is video 25 capturing. The overlay function is only available after calling 26 the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. 27 28The driver may support simultaneous overlay and capturing using the 29read/write and streaming I/O methods. If so, operation at the nominal 30frame rate of the video standard is not guaranteed. Frames may be 31directed away from overlay to capture, or one field may be used for 32overlay and the other for capture if the capture parameters permit this. 33 34Applications should use different file descriptors for capturing and 35overlay. This must be supported by all drivers capable of simultaneous 36capturing and overlay. Optionally these drivers may also permit 37capturing and overlay with a single file descriptor for compatibility 38with V4L and earlier versions of V4L2. [#f1]_ 39 40 41Querying Capabilities 42===================== 43 44Devices supporting the video overlay interface set the 45``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct 46:c:type:`v4l2_capability` returned by the 47:ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O 48method specified below must be supported. Tuners and audio inputs are 49optional. 50 51 52Supplemental Functions 53====================== 54 55Video overlay devices shall support :ref:`audio input <audio>`, 56:ref:`tuner`, :ref:`controls <control>`, 57:ref:`cropping and scaling <crop>` and 58:ref:`streaming parameter <streaming-par>` ioctls as needed. The 59:ref:`video input <video>` and :ref:`video standard <standard>` 60ioctls must be supported by all video overlay devices. 61 62 63Setup 64===== 65 66Before overlay can commence applications must program the driver with 67frame buffer parameters, namely the address and size of the frame buffer 68and the image format, for example RGB 5:6:5. The 69:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and 70:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and 71set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is 72privileged because it allows to set up DMA into physical memory, 73bypassing the memory protection mechanisms of the kernel. Only the 74superuser can change the frame buffer address and size. Users are not 75supposed to run TV applications as root or with SUID bit set. A small 76helper application with suitable privileges should query the graphics 77system and program the V4L2 driver at the appropriate time. 78 79Some devices add the video overlay to the output signal of the graphics 80card. In this case the frame buffer is not modified by the video device, 81and the frame buffer address and pixel format are not needed by the 82driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application 83can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` 84ioctl. 85 86A driver may support any (or none) of five clipping/blending methods: 87 881. Chroma-keying displays the overlaid image only where pixels in the 89 primary graphics surface assume a certain color. 90 912. A bitmap can be specified where each bit corresponds to a pixel in 92 the overlaid image. When the bit is set, the corresponding video 93 pixel is displayed, otherwise a pixel of the graphics surface. 94 953. A list of clipping rectangles can be specified. In these regions *no* 96 video is displayed, so the graphics surface can be seen here. 97 984. The framebuffer has an alpha channel that can be used to clip or 99 blend the framebuffer with the video. 100 1015. A global alpha value can be specified to blend the framebuffer 102 contents with video images. 103 104When simultaneous capturing and overlay is supported and the hardware 105prohibits different image and frame buffer formats, the format requested 106first takes precedence. The attempt to capture 107(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay 108(:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error 109code or return accordingly modified parameters.. 110 111 112Overlay Window 113============== 114 115The overlaid image is determined by cropping and overlay window 116parameters. The former select an area of the video picture to capture, 117the latter how images are overlaid and clipped. Cropping initialization 118at minimum requires to reset the parameters to defaults. An example is 119given in :ref:`crop`. 120 121The overlay window is described by a struct 122:c:type:`v4l2_window`. It defines the size of the image, 123its position over the graphics surface and the clipping to be applied. 124To get the current parameters applications set the ``type`` field of a 125struct :c:type:`v4l2_format` to 126``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the 127:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the 128struct :c:type:`v4l2_window` substructure named ``win``. It is not 129possible to retrieve a previously programmed clipping list or bitmap. 130 131To program the overlay window applications set the ``type`` field of a 132struct :c:type:`v4l2_format` to 133``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and 134call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver 135adjusts the parameters against hardware limits and returns the actual 136parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the 137:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn 138about driver capabilities without actually changing driver state. Unlike 139:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled. 140 141The scaling factor of the overlaid image is implied by the width and 142height given in struct :c:type:`v4l2_window` and the size 143of the cropping rectangle. For more information see :ref:`crop`. 144 145When simultaneous capturing and overlay is supported and the hardware 146prohibits different image and window sizes, the size requested first 147takes precedence. The attempt to capture or overlay as well 148(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error 149code or return accordingly modified parameters. 150 151 152.. c:type:: v4l2_window 153 154struct v4l2_window 155------------------ 156 157``struct v4l2_rect w`` 158 Size and position of the window relative to the top, left corner of 159 the frame buffer defined with 160 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the 161 frame buffer width and height, the ``x`` and ``y`` coordinates can 162 be negative, and it can lie completely outside the frame buffer. The 163 driver clips the window accordingly, or if that is not possible, 164 modifies its size and/or position. 165 166``enum v4l2_field field`` 167 Applications set this field to determine which video field shall be 168 overlaid, typically one of ``V4L2_FIELD_ANY`` (0), 169 ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or 170 ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different 171 field order and return the actual setting here. 172 173``__u32 chromakey`` 174 When chroma-keying has been negotiated with 175 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field 176 to the desired pixel value for the chroma key. The format is the 177 same as the pixel format of the framebuffer (struct 178 :c:type:`v4l2_framebuffer` ``fmt.pixelformat`` 179 field), with bytes in host order. E. g. for 180 :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should 181 be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host. 182 183``struct v4l2_clip * clips`` 184 When chroma-keying has *not* been negotiated and 185 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 186 applications can set this field to point to an array of clipping 187 rectangles. 188 189 Like the window coordinates w, clipping rectangles are defined 190 relative to the top, left corner of the frame buffer. However 191 clipping rectangles must not extend the frame buffer width and 192 height, and they must not overlap. If possible applications 193 should merge adjacent rectangles. Whether this must create 194 x-y or y-x bands, or the order of rectangles, is not defined. When 195 clip lists are not supported the driver ignores this field. Its 196 contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` 197 are undefined. 198 199``__u32 clipcount`` 200 When the application set the ``clips`` field, this field must 201 contain the number of clipping rectangles in the list. When clip 202 lists are not supported the driver ignores this field, its contents 203 after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are 204 supported but no clipping is desired this field must be set to zero. 205 206``void * bitmap`` 207 When chroma-keying has *not* been negotiated and 208 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 209 applications can set this field to point to a clipping bit mask. 210 211It must be of the same size as the window, ``w.width`` and ``w.height``. 212Each bit corresponds to a pixel in the overlaid image, which is 213displayed only when the bit is *set*. Pixel coordinates translate to 214bits like: 215 216 217.. code-block:: c 218 219 ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7)) 220 221where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_ 222 223When a clipping bit mask is not supported the driver ignores this field, 224its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are 225undefined. When a bit mask is supported but no clipping is desired this 226field must be set to ``NULL``. 227 228Applications need not create a clip list or bit mask. When they pass 229both, or despite negotiating chroma-keying, the results are undefined. 230Regardless of the chosen method, the clipping abilities of the hardware 231may be limited in quantity or quality. The results when these limits are 232exceeded are undefined. [#f3]_ 233 234``__u8 global_alpha`` 235 The global alpha value used to blend the framebuffer with video 236 images, if global alpha blending has been negotiated 237 (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see 238 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`, 239 :ref:`framebuffer-flags`). 240 241.. note:: 242 243 This field was added in Linux 2.6.23, extending the 244 structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>` 245 ioctls, which take a pointer to a :c:type:`v4l2_format` 246 parent structure with padding bytes at the end, are not affected. 247 248 249.. c:type:: v4l2_clip 250 251struct v4l2_clip [#f4]_ 252----------------------- 253 254``struct v4l2_rect c`` 255 Coordinates of the clipping rectangle, relative to the top, left 256 corner of the frame buffer. Only window pixels *outside* all 257 clipping rectangles are displayed. 258 259``struct v4l2_clip * next`` 260 Pointer to the next clipping rectangle, ``NULL`` when this is the last 261 rectangle. Drivers ignore this field, it cannot be used to pass a 262 linked list of clipping rectangles. 263 264 265.. c:type:: v4l2_rect 266 267struct v4l2_rect 268---------------- 269 270``__s32 left`` 271 Horizontal offset of the top, left corner of the rectangle, in 272 pixels. 273 274``__s32 top`` 275 Vertical offset of the top, left corner of the rectangle, in pixels. 276 Offsets increase to the right and down. 277 278``__u32 width`` 279 Width of the rectangle, in pixels. 280 281``__u32 height`` 282 Height of the rectangle, in pixels. 283 284 285Enabling Overlay 286================ 287 288To start or stop the frame buffer overlay applications call the 289:ref:`VIDIOC_OVERLAY` ioctl. 290 291.. [#f1] 292 A common application of two file descriptors is the XFree86 293 :ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application. 294 While the X server controls video overlay, the application can take 295 advantage of memory mapping and DMA. 296 297 In the opinion of the designers of this API, no driver writer taking 298 the efforts to support simultaneous capturing and overlay will 299 restrict this ability by requiring a single file descriptor, as in 300 V4L and earlier versions of V4L2. Making this optional means 301 applications depending on two file descriptors need backup routines 302 to be compatible with all drivers, which is considerable more work 303 than using two fds in applications which do not. Also two fd's fit 304 the general concept of one file descriptor for each logical stream. 305 Hence as a complexity trade-off drivers *must* support two file 306 descriptors and *may* support single fd operation. 307 308.. [#f2] 309 Should we require ``w.width`` to be a multiple of eight? 310 311.. [#f3] 312 When the image is written into frame buffer memory it will be 313 undesirable if the driver clips out less pixels than expected, 314 because the application and graphics system are not aware these 315 regions need to be refreshed. The driver should clip out more pixels 316 or not write the image at all. 317 318.. [#f4] 319 The X Window system defines "regions" which are vectors of ``struct 320 BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and 321 ``height = y2 - y1``, so one cannot pass X11 clip lists directly. 322