1.. Permission is granted to copy, distribute and/or modify this 2.. document under the terms of the GNU Free Documentation License, 3.. Version 1.1 or any later version published by the Free Software 4.. Foundation, with no Invariant Sections, no Front-Cover Texts 5.. and no Back-Cover Texts. A copy of the license is included at 6.. Documentation/userspace-api/media/fdl-appendix.rst. 7.. 8.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections 9 10.. _overlay: 11 12*********************** 13Video Overlay Interface 14*********************** 15 16**Also known as Framebuffer Overlay or Previewing.** 17 18Video overlay devices have the ability to genlock (TV-)video into the 19(VGA-)video signal of a graphics card, or to store captured images 20directly in video memory of a graphics card, typically with clipping. 21This can be considerable more efficient than capturing images and 22displaying them by other means. In the old days when only nuclear power 23plants needed cooling towers this used to be the only way to put live 24video into a window. 25 26Video overlay devices are accessed through the same character special 27files as :ref:`video capture <capture>` devices. 28 29.. note:: 30 31 The default function of a ``/dev/video`` device is video 32 capturing. The overlay function is only available after calling 33 the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. 34 35The driver may support simultaneous overlay and capturing using the 36read/write and streaming I/O methods. If so, operation at the nominal 37frame rate of the video standard is not guaranteed. Frames may be 38directed away from overlay to capture, or one field may be used for 39overlay and the other for capture if the capture parameters permit this. 40 41Applications should use different file descriptors for capturing and 42overlay. This must be supported by all drivers capable of simultaneous 43capturing and overlay. Optionally these drivers may also permit 44capturing and overlay with a single file descriptor for compatibility 45with V4L and earlier versions of V4L2. [#f1]_ 46 47 48Querying Capabilities 49===================== 50 51Devices supporting the video overlay interface set the 52``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct 53:c:type:`v4l2_capability` returned by the 54:ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O 55method specified below must be supported. Tuners and audio inputs are 56optional. 57 58 59Supplemental Functions 60====================== 61 62Video overlay devices shall support :ref:`audio input <audio>`, 63:ref:`tuner`, :ref:`controls <control>`, 64:ref:`cropping and scaling <crop>` and 65:ref:`streaming parameter <streaming-par>` ioctls as needed. The 66:ref:`video input <video>` and :ref:`video standard <standard>` 67ioctls must be supported by all video overlay devices. 68 69 70Setup 71===== 72 73Before overlay can commence applications must program the driver with 74frame buffer parameters, namely the address and size of the frame buffer 75and the image format, for example RGB 5:6:5. The 76:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and 77:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and 78set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is 79privileged because it allows to set up DMA into physical memory, 80bypassing the memory protection mechanisms of the kernel. Only the 81superuser can change the frame buffer address and size. Users are not 82supposed to run TV applications as root or with SUID bit set. A small 83helper application with suitable privileges should query the graphics 84system and program the V4L2 driver at the appropriate time. 85 86Some devices add the video overlay to the output signal of the graphics 87card. In this case the frame buffer is not modified by the video device, 88and the frame buffer address and pixel format are not needed by the 89driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application 90can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` 91ioctl. 92 93A driver may support any (or none) of five clipping/blending methods: 94 951. Chroma-keying displays the overlaid image only where pixels in the 96 primary graphics surface assume a certain color. 97 982. A bitmap can be specified where each bit corresponds to a pixel in 99 the overlaid image. When the bit is set, the corresponding video 100 pixel is displayed, otherwise a pixel of the graphics surface. 101 1023. A list of clipping rectangles can be specified. In these regions *no* 103 video is displayed, so the graphics surface can be seen here. 104 1054. The framebuffer has an alpha channel that can be used to clip or 106 blend the framebuffer with the video. 107 1085. A global alpha value can be specified to blend the framebuffer 109 contents with video images. 110 111When simultaneous capturing and overlay is supported and the hardware 112prohibits different image and frame buffer formats, the format requested 113first takes precedence. The attempt to capture 114(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay 115(:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error 116code or return accordingly modified parameters.. 117 118 119Overlay Window 120============== 121 122The overlaid image is determined by cropping and overlay window 123parameters. The former select an area of the video picture to capture, 124the latter how images are overlaid and clipped. Cropping initialization 125at minimum requires to reset the parameters to defaults. An example is 126given in :ref:`crop`. 127 128The overlay window is described by a struct 129:c:type:`v4l2_window`. It defines the size of the image, 130its position over the graphics surface and the clipping to be applied. 131To get the current parameters applications set the ``type`` field of a 132struct :c:type:`v4l2_format` to 133``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the 134:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the 135struct :c:type:`v4l2_window` substructure named ``win``. It is not 136possible to retrieve a previously programmed clipping list or bitmap. 137 138To program the overlay window applications set the ``type`` field of a 139struct :c:type:`v4l2_format` to 140``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and 141call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver 142adjusts the parameters against hardware limits and returns the actual 143parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the 144:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn 145about driver capabilities without actually changing driver state. Unlike 146:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled. 147 148The scaling factor of the overlaid image is implied by the width and 149height given in struct :c:type:`v4l2_window` and the size 150of the cropping rectangle. For more information see :ref:`crop`. 151 152When simultaneous capturing and overlay is supported and the hardware 153prohibits different image and window sizes, the size requested first 154takes precedence. The attempt to capture or overlay as well 155(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error 156code or return accordingly modified parameters. 157 158 159.. c:type:: v4l2_window 160 161struct v4l2_window 162------------------ 163 164``struct v4l2_rect w`` 165 Size and position of the window relative to the top, left corner of 166 the frame buffer defined with 167 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the 168 frame buffer width and height, the ``x`` and ``y`` coordinates can 169 be negative, and it can lie completely outside the frame buffer. The 170 driver clips the window accordingly, or if that is not possible, 171 modifies its size and/or position. 172 173``enum v4l2_field field`` 174 Applications set this field to determine which video field shall be 175 overlaid, typically one of ``V4L2_FIELD_ANY`` (0), 176 ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or 177 ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different 178 field order and return the actual setting here. 179 180``__u32 chromakey`` 181 When chroma-keying has been negotiated with 182 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field 183 to the desired pixel value for the chroma key. The format is the 184 same as the pixel format of the framebuffer (struct 185 :c:type:`v4l2_framebuffer` ``fmt.pixelformat`` 186 field), with bytes in host order. E. g. for 187 :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should 188 be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host. 189 190``struct v4l2_clip * clips`` 191 When chroma-keying has *not* been negotiated and 192 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 193 applications can set this field to point to an array of clipping 194 rectangles. 195 196 Like the window coordinates w, clipping rectangles are defined 197 relative to the top, left corner of the frame buffer. However 198 clipping rectangles must not extend the frame buffer width and 199 height, and they must not overlap. If possible applications 200 should merge adjacent rectangles. Whether this must create 201 x-y or y-x bands, or the order of rectangles, is not defined. When 202 clip lists are not supported the driver ignores this field. Its 203 contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` 204 are undefined. 205 206``__u32 clipcount`` 207 When the application set the ``clips`` field, this field must 208 contain the number of clipping rectangles in the list. When clip 209 lists are not supported the driver ignores this field, its contents 210 after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are 211 supported but no clipping is desired this field must be set to zero. 212 213``void * bitmap`` 214 When chroma-keying has *not* been negotiated and 215 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 216 applications can set this field to point to a clipping bit mask. 217 218It must be of the same size as the window, ``w.width`` and ``w.height``. 219Each bit corresponds to a pixel in the overlaid image, which is 220displayed only when the bit is *set*. Pixel coordinates translate to 221bits like: 222 223 224.. code-block:: c 225 226 ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7)) 227 228where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_ 229 230When a clipping bit mask is not supported the driver ignores this field, 231its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are 232undefined. When a bit mask is supported but no clipping is desired this 233field must be set to ``NULL``. 234 235Applications need not create a clip list or bit mask. When they pass 236both, or despite negotiating chroma-keying, the results are undefined. 237Regardless of the chosen method, the clipping abilities of the hardware 238may be limited in quantity or quality. The results when these limits are 239exceeded are undefined. [#f3]_ 240 241``__u8 global_alpha`` 242 The global alpha value used to blend the framebuffer with video 243 images, if global alpha blending has been negotiated 244 (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see 245 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`, 246 :ref:`framebuffer-flags`). 247 248.. note:: 249 250 This field was added in Linux 2.6.23, extending the 251 structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>` 252 ioctls, which take a pointer to a :c:type:`v4l2_format` 253 parent structure with padding bytes at the end, are not affected. 254 255 256.. c:type:: v4l2_clip 257 258struct v4l2_clip [#f4]_ 259----------------------- 260 261``struct v4l2_rect c`` 262 Coordinates of the clipping rectangle, relative to the top, left 263 corner of the frame buffer. Only window pixels *outside* all 264 clipping rectangles are displayed. 265 266``struct v4l2_clip * next`` 267 Pointer to the next clipping rectangle, ``NULL`` when this is the last 268 rectangle. Drivers ignore this field, it cannot be used to pass a 269 linked list of clipping rectangles. 270 271 272.. c:type:: v4l2_rect 273 274struct v4l2_rect 275---------------- 276 277``__s32 left`` 278 Horizontal offset of the top, left corner of the rectangle, in 279 pixels. 280 281``__s32 top`` 282 Vertical offset of the top, left corner of the rectangle, in pixels. 283 Offsets increase to the right and down. 284 285``__u32 width`` 286 Width of the rectangle, in pixels. 287 288``__u32 height`` 289 Height of the rectangle, in pixels. 290 291 292Enabling Overlay 293================ 294 295To start or stop the frame buffer overlay applications call the 296:ref:`VIDIOC_OVERLAY` ioctl. 297 298.. [#f1] 299 A common application of two file descriptors is the XFree86 300 :ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application. 301 While the X server controls video overlay, the application can take 302 advantage of memory mapping and DMA. 303 304 In the opinion of the designers of this API, no driver writer taking 305 the efforts to support simultaneous capturing and overlay will 306 restrict this ability by requiring a single file descriptor, as in 307 V4L and earlier versions of V4L2. Making this optional means 308 applications depending on two file descriptors need backup routines 309 to be compatible with all drivers, which is considerable more work 310 than using two fds in applications which do not. Also two fd's fit 311 the general concept of one file descriptor for each logical stream. 312 Hence as a complexity trade-off drivers *must* support two file 313 descriptors and *may* support single fd operation. 314 315.. [#f2] 316 Should we require ``w.width`` to be a multiple of eight? 317 318.. [#f3] 319 When the image is written into frame buffer memory it will be 320 undesirable if the driver clips out less pixels than expected, 321 because the application and graphics system are not aware these 322 regions need to be refreshed. The driver should clip out more pixels 323 or not write the image at all. 324 325.. [#f4] 326 The X Window system defines "regions" which are vectors of ``struct 327 BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and 328 ``height = y2 - y1``, so one cannot pass X11 clip lists directly. 329