Documentation/linux_tv/media/v4l/dev-overlay.rst

   1 .. -*- coding: utf-8; mode: rst -*-
   2
   3 .. _overlay:
   4
   5 ***********************
   6 Video Overlay Interface
   7 ***********************
   8
   9
  10 **Also known as Framebuffer Overlay or Previewing**
  11 Video overlay devices have the ability to genlock (TV-)video into the
  12 (VGA-)video signal of a graphics card, or to store captured images
  13 directly in video memory of a graphics card, typically with clipping.
  14 This can be considerable more efficient than capturing images and
  15 displaying them by other means. In the old days when only nuclear power
  16 plants needed cooling towers this used to be the only way to put live
  17 video into a window.
  18
  19 Video overlay devices are accessed through the same character special
  20 files as :ref:`video capture <capture>` devices. Note the default
  21 function of a ``/dev/video`` device is video capturing. The overlay
  22 function is only available after calling the
  23 :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl.
  24
  25 The driver may support simultaneous overlay and capturing using the
  26 read/write and streaming I/O methods. If so, operation at the nominal
  27 frame rate of the video standard is not guaranteed. Frames may be
  28 directed away from overlay to capture, or one field may be used for
  29 overlay and the other for capture if the capture parameters permit this.
  30
  31 Applications should use different file descriptors for capturing and
  32 overlay. This must be supported by all drivers capable of simultaneous
  33 capturing and overlay. Optionally these drivers may also permit
  34 capturing and overlay with a single file descriptor for compatibility
  35 with V4L and earlier versions of V4L2. [1]_
  36
  37
  38 Querying Capabilities
  39 =====================
  40
  41 Devices supporting the video overlay interface set the
  42 ``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct
  43 :ref:`v4l2_capability <v4l2-capability>` returned by the
  44 :ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O
  45 method specified below must be supported. Tuners and audio inputs are
  46 optional.
  47
  48
  49 Supplemental Functions
  50 ======================
  51
  52 Video overlay devices shall support :ref:`audio input <audio>`,
  53 :ref:`tuner`, :ref:`controls <control>`,
  54 :ref:`cropping and scaling <crop>` and
  55 :ref:`streaming parameter <streaming-par>` ioctls as needed. The
  56 :ref:`video input <video>` and :ref:`video standard <standard>`
  57 ioctls must be supported by all video overlay devices.
  58
  59
  60 Setup
  61 =====
  62
  63 Before overlay can commence applications must program the driver with
  64 frame buffer parameters, namely the address and size of the frame buffer
  65 and the image format, for example RGB 5:6:5. The
  66 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and
  67 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and
  68 set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is
  69 privileged because it allows to set up DMA into physical memory,
  70 bypassing the memory protection mechanisms of the kernel. Only the
  71 superuser can change the frame buffer address and size. Users are not
  72 supposed to run TV applications as root or with SUID bit set. A small
  73 helper application with suitable privileges should query the graphics
  74 system and program the V4L2 driver at the appropriate time.
  75
  76 Some devices add the video overlay to the output signal of the graphics
  77 card. In this case the frame buffer is not modified by the video device,
  78 and the frame buffer address and pixel format are not needed by the
  79 driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application
  80 can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`
  81 ioctl.
  82
  83 A driver may support any (or none) of five clipping/blending methods:
  84
  85 1. Chroma-keying displays the overlaid image only where pixels in the
  86    primary graphics surface assume a certain color.
  87
  88 2. A bitmap can be specified where each bit corresponds to a pixel in
  89    the overlaid image. When the bit is set, the corresponding video
  90    pixel is displayed, otherwise a pixel of the graphics surface.
  91
  92 3. A list of clipping rectangles can be specified. In these regions *no*
  93    video is displayed, so the graphics surface can be seen here.
  94
  95 4. The framebuffer has an alpha channel that can be used to clip or
  96    blend the framebuffer with the video.
  97
  98 5. A global alpha value can be specified to blend the framebuffer
  99    contents with video images.
 100
 101 When simultaneous capturing and overlay is supported and the hardware
 102 prohibits different image and frame buffer formats, the format requested
 103 first takes precedence. The attempt to capture
 104 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay
 105 (:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an EBUSY error
 106 code or return accordingly modified parameters..
 107
 108
 109 Overlay Window
 110 ==============
 111
 112 The overlaid image is determined by cropping and overlay window
 113 parameters. The former select an area of the video picture to capture,
 114 the latter how images are overlaid and clipped. Cropping initialization
 115 at minimum requires to reset the parameters to defaults. An example is
 116 given in :ref:`crop`.
 117
 118 The overlay window is described by a struct
 119 :ref:`v4l2_window <v4l2-window>`. It defines the size of the image,
 120 its position over the graphics surface and the clipping to be applied.
 121 To get the current parameters applications set the ``type`` field of a
 122 struct :ref:`v4l2_format <v4l2-format>` to
 123 ``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the
 124 :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the
 125 :c:type:`struct v4l2_window` substructure named ``win``. It is not
 126 possible to retrieve a previously programmed clipping list or bitmap.
 127
 128 To program the overlay window applications set the ``type`` field of a
 129 struct :ref:`v4l2_format <v4l2-format>` to
 130 ``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and
 131 call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver
 132 adjusts the parameters against hardware limits and returns the actual
 133 parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the
 134 :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn
 135 about driver capabilities without actually changing driver state. Unlike
 136 :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled.
 137
 138 The scaling factor of the overlaid image is implied by the width and
 139 height given in struct :ref:`v4l2_window <v4l2-window>` and the size
 140 of the cropping rectangle. For more information see :ref:`crop`.
 141
 142 When simultaneous capturing and overlay is supported and the hardware
 143 prohibits different image and window sizes, the size requested first
 144 takes precedence. The attempt to capture or overlay as well
 145 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an EBUSY error
 146 code or return accordingly modified parameters.
 147
 148
 149 .. _v4l2-window:
 150
 151 struct v4l2_window
 152 ------------------
 153
 154 ``struct v4l2_rect w``
 155     Size and position of the window relative to the top, left corner of
 156     the frame buffer defined with
 157     :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the
 158     frame buffer width and height, the ``x`` and ``y`` coordinates can
 159     be negative, and it can lie completely outside the frame buffer. The
 160     driver clips the window accordingly, or if that is not possible,
 161     modifies its size and/or position.
 162
 163 ``enum v4l2_field field``
 164     Applications set this field to determine which video field shall be
 165     overlaid, typically one of ``V4L2_FIELD_ANY`` (0),
 166     ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or
 167     ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different
 168     field order and return the actual setting here.
 169
 170 ``__u32 chromakey``
 171     When chroma-keying has been negotiated with
 172     :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field
 173     to the desired pixel value for the chroma key. The format is the
 174     same as the pixel format of the framebuffer (struct
 175     :ref:`v4l2_framebuffer <v4l2-framebuffer>` ``fmt.pixelformat``
 176     field), with bytes in host order. E. g. for
 177     :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should
 178     be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host.
 179
 180 ``struct v4l2_clip * clips``
 181     When chroma-keying has *not* been negotiated and
 182     :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
 183     applications can set this field to point to an array of clipping
 184     rectangles.
 185
 186 Like the window coordinates
 187 w
 188 , clipping rectangles are defined relative to the top, left corner of
 189 the frame buffer. However clipping rectangles must not extend the frame
 190 buffer width and height, and they must not overlap. If possible
 191 applications should merge adjacent rectangles. Whether this must create
 192 x-y or y-x bands, or the order of rectangles, is not defined. When clip
 193 lists are not supported the driver ignores this field. Its contents
 194 after calling
 195 !ri!:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
 196 are undefined.
 197
 198 ``__u32 clipcount``
 199     When the application set the ``clips`` field, this field must
 200     contain the number of clipping rectangles in the list. When clip
 201     lists are not supported the driver ignores this field, its contents
 202     after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are
 203     supported but no clipping is desired this field must be set to zero.
 204
 205 ``void * bitmap``
 206     When chroma-keying has *not* been negotiated and
 207     :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
 208     applications can set this field to point to a clipping bit mask.
 209
 210 It must be of the same size as the window, ``w.width`` and ``w.height``.
 211 Each bit corresponds to a pixel in the overlaid image, which is
 212 displayed only when the bit is *set*. Pixel coordinates translate to
 213 bits like:
 214
 215
 216 .. code-block:: c
 217
 218     ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7))
 219
 220 where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [2]_
 221
 222 When a clipping bit mask is not supported the driver ignores this field,
 223 its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are
 224 undefined. When a bit mask is supported but no clipping is desired this
 225 field must be set to ``NULL``.
 226
 227 Applications need not create a clip list or bit mask. When they pass
 228 both, or despite negotiating chroma-keying, the results are undefined.
 229 Regardless of the chosen method, the clipping abilities of the hardware
 230 may be limited in quantity or quality. The results when these limits are
 231 exceeded are undefined. [3]_
 232
 233 ``__u8 global_alpha``
 234     The global alpha value used to blend the framebuffer with video
 235     images, if global alpha blending has been negotiated
 236     (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see
 237     :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`,
 238     :ref:`framebuffer-flags`).
 239
 240 Note this field was added in Linux 2.6.23, extending the structure.
 241 However the
 242 !ri!:ref:`VIDIOC_G/S/TRY_FMT <VIDIOC_G_FMT>`
 243 ioctls, which take a pointer to a
 244 !ri!:ref:`v4l2_format <v4l2-format>`
 245 parent structure with padding bytes at the end, are not affected.
 246
 247
 248 .. _v4l2-clip:
 249
 250 struct v4l2_clip
 251 ----------------
 252
 253 The X Window system defines "regions" which are vectors of struct BoxRec
 254 { short x1, y1, x2, y2; } with width = x2 - x1 and height = y2 - y1, so
 255 one cannot pass X11 clip lists directly.
 256
 257 ``struct v4l2_rect c``
 258     Coordinates of the clipping rectangle, relative to the top, left
 259     corner of the frame buffer. Only window pixels *outside* all
 260     clipping rectangles are displayed.
 261
 262 ``struct v4l2_clip * next``
 263     Pointer to the next clipping rectangle, NULL when this is the last
 264     rectangle. Drivers ignore this field, it cannot be used to pass a
 265     linked list of clipping rectangles.
 266
 267
 268 .. _v4l2-rect:
 269
 270 struct v4l2_rect
 271 ----------------
 272
 273 ``__s32 left``
 274     Horizontal offset of the top, left corner of the rectangle, in
 275     pixels.
 276
 277 ``__s32 top``
 278     Vertical offset of the top, left corner of the rectangle, in pixels.
 279     Offsets increase to the right and down.
 280
 281 ``__u32 width``
 282     Width of the rectangle, in pixels.
 283
 284 ``__u32 height``
 285     Height of the rectangle, in pixels.
 286
 287
 288 Enabling Overlay
 289 ================
 290
 291 To start or stop the frame buffer overlay applications call the
 292 :ref:`VIDIOC_OVERLAY` ioctl.
 293
 294 .. [1]
 295    A common application of two file descriptors is the XFree86
 296    :ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application.
 297    While the X server controls video overlay, the application can take
 298    advantage of memory mapping and DMA.
 299
 300    In the opinion of the designers of this API, no driver writer taking
 301    the efforts to support simultaneous capturing and overlay will
 302    restrict this ability by requiring a single file descriptor, as in
 303    V4L and earlier versions of V4L2. Making this optional means
 304    applications depending on two file descriptors need backup routines
 305    to be compatible with all drivers, which is considerable more work
 306    than using two fds in applications which do not. Also two fd's fit
 307    the general concept of one file descriptor for each logical stream.
 308    Hence as a complexity trade-off drivers *must* support two file
 309    descriptors and *may* support single fd operation.
 310
 311 .. [2]
 312    Should we require ``w.width`` to be a multiple of eight?
 313
 314 .. [3]
 315    When the image is written into frame buffer memory it will be
 316    undesirable if the driver clips out less pixels than expected,
 317    because the application and graphics system are not aware these
 318    regions need to be refreshed. The driver should clip out more pixels
 319    or not write the image at all.
 320
 321
 322 .. ------------------------------------------------------------------------------
 323 .. This file was automatically converted from DocBook-XML with the dbxml
 324 .. library (https://github.com/return42/sphkerneldoc). The origin XML comes
 325 .. from the linux kernel, refer to:
 326 ..
 327 .. * https://github.com/torvalds/linux/tree/master/Documentation/DocBook
 328 .. ------------------------------------------------------------------------------