321 lines
13 KiB
ReStructuredText
321 lines
13 KiB
ReStructuredText
|
.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
|
||
|
|
|
||
|
|
.. _overlay:
|
||
|
|
|
||
|
|
***********************
|
||
|
|
Video Overlay Interface
|
||
|
|
***********************
|
||
|
|
|
||
|
|
**Also known as Framebuffer Overlay or Previewing.**
|
||
|
|
|
||
|
|
Video overlay devices have the ability to genlock (TV-)video into the
|
||
|
|
(VGA-)video signal of a graphics card, or to store captured images
|
||
|
|
directly in video memory of a graphics card, typically with clipping.
|
||
|
|
This can be considerable more efficient than capturing images and
|
||
|
|
displaying them by other means. In the old days when only nuclear power
|
||
|
|
plants needed cooling towers this used to be the only way to put live
|
||
|
|
video into a window.
|
||
|
|
|
||
|
|
Video overlay devices are accessed through the same character special
|
||
|
|
files as :ref:`video capture <capture>` devices.
|
||
|
|
|
||
|
|
.. note::
|
||
|
|
|
||
|
|
The default function of a ``/dev/video`` device is video
|
||
|
|
capturing. The overlay function is only available after calling
|
||
|
|
the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl.
|
||
|
|
|
||
|
|
The driver may support simultaneous overlay and capturing using the
|
||
|
|
read/write and streaming I/O methods. If so, operation at the nominal
|
||
|
|
frame rate of the video standard is not guaranteed. Frames may be
|
||
|
|
directed away from overlay to capture, or one field may be used for
|
||
|
|
overlay and the other for capture if the capture parameters permit this.
|
||
|
|
|
||
|
|
Applications should use different file descriptors for capturing and
|
||
|
|
overlay. This must be supported by all drivers capable of simultaneous
|
||
|
|
capturing and overlay. Optionally these drivers may also permit
|
||
|
|
capturing and overlay with a single file descriptor for compatibility
|
||
|
|
with V4L and earlier versions of V4L2. [#f1]_
|
||
|
|
|
||
|
|
A common application of two file descriptors is the X11
|
||
|
|
:ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application.
|
||
|
|
While the X server controls video overlay, the application can take
|
||
|
|
advantage of memory mapping and DMA.
|
||
|
|
|
||
|
|
Querying Capabilities
|
||
|
|
=====================
|
||
|
|
|
||
|
|
Devices supporting the video overlay interface set the
|
||
|
|
``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct
|
||
|
|
:c:type:`v4l2_capability` returned by the
|
||
|
|
:ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O
|
||
|
|
method specified below must be supported. Tuners and audio inputs are
|
||
|
|
optional.
|
||
|
|
|
||
|
|
|
||
|
|
Supplemental Functions
|
||
|
|
======================
|
||
|
|
|
||
|
|
Video overlay devices shall support :ref:`audio input <audio>`,
|
||
|
|
:ref:`tuner`, :ref:`controls <control>`,
|
||
|
|
:ref:`cropping and scaling <crop>` and
|
||
|
|
:ref:`streaming parameter <streaming-par>` ioctls as needed. The
|
||
|
|
:ref:`video input <video>` and :ref:`video standard <standard>`
|
||
|
|
ioctls must be supported by all video overlay devices.
|
||
|
|
|
||
|
|
|
||
|
|
Setup
|
||
|
|
=====
|
||
|
|
|
||
|
|
Before overlay can commence applications must program the driver with
|
||
|
|
frame buffer parameters, namely the address and size of the frame buffer
|
||
|
|
and the image format, for example RGB 5:6:5. The
|
||
|
|
:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and
|
||
|
|
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and
|
||
|
|
set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is
|
||
|
|
privileged because it allows to set up DMA into physical memory,
|
||
|
|
bypassing the memory protection mechanisms of the kernel. Only the
|
||
|
|
superuser can change the frame buffer address and size. Users are not
|
||
|
|
supposed to run TV applications as root or with SUID bit set. A small
|
||
|
|
helper application with suitable privileges should query the graphics
|
||
|
|
system and program the V4L2 driver at the appropriate time.
|
||
|
|
|
||
|
|
Some devices add the video overlay to the output signal of the graphics
|
||
|
|
card. In this case the frame buffer is not modified by the video device,
|
||
|
|
and the frame buffer address and pixel format are not needed by the
|
||
|
|
driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application
|
||
|
|
can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`
|
||
|
|
ioctl.
|
||
|
|
|
||
|
|
A driver may support any (or none) of five clipping/blending methods:
|
||
|
|
|
||
|
|
1. Chroma-keying displays the overlaid image only where pixels in the
|
||
|
|
primary graphics surface assume a certain color.
|
||
|
|
|
||
|
|
2. A bitmap can be specified where each bit corresponds to a pixel in
|
||
|
|
the overlaid image. When the bit is set, the corresponding video
|
||
|
|
pixel is displayed, otherwise a pixel of the graphics surface.
|
||
|
|
|
||
|
|
3. A list of clipping rectangles can be specified. In these regions *no*
|
||
|
|
video is displayed, so the graphics surface can be seen here.
|
||
|
|
|
||
|
|
4. The framebuffer has an alpha channel that can be used to clip or
|
||
|
|
blend the framebuffer with the video.
|
||
|
|
|
||
|
|
5. A global alpha value can be specified to blend the framebuffer
|
||
|
|
contents with video images.
|
||
|
|
|
||
|
|
When simultaneous capturing and overlay is supported and the hardware
|
||
|
|
prohibits different image and frame buffer formats, the format requested
|
||
|
|
first takes precedence. The attempt to capture
|
||
|
|
(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay
|
||
|
|
(:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error
|
||
|
|
code or return accordingly modified parameters..
|
||
|
|
|
||
|
|
|
||
|
|
Overlay Window
|
||
|
|
==============
|
||
|
|
|
||
|
|
The overlaid image is determined by cropping and overlay window
|
||
|
|
parameters. The former select an area of the video picture to capture,
|
||
|
|
the latter how images are overlaid and clipped. Cropping initialization
|
||
|
|
at minimum requires to reset the parameters to defaults. An example is
|
||
|
|
given in :ref:`crop`.
|
||
|
|
|
||
|
|
The overlay window is described by a struct
|
||
|
|
:c:type:`v4l2_window`. It defines the size of the image,
|
||
|
|
its position over the graphics surface and the clipping to be applied.
|
||
|
|
To get the current parameters applications set the ``type`` field of a
|
||
|
|
struct :c:type:`v4l2_format` to
|
||
|
|
``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the
|
||
|
|
:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the
|
||
|
|
struct :c:type:`v4l2_window` substructure named ``win``. It is not
|
||
|
|
possible to retrieve a previously programmed clipping list or bitmap.
|
||
|
|
|
||
|
|
To program the overlay window applications set the ``type`` field of a
|
||
|
|
struct :c:type:`v4l2_format` to
|
||
|
|
``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and
|
||
|
|
call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver
|
||
|
|
adjusts the parameters against hardware limits and returns the actual
|
||
|
|
parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the
|
||
|
|
:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn
|
||
|
|
about driver capabilities without actually changing driver state. Unlike
|
||
|
|
:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled.
|
||
|
|
|
||
|
|
The scaling factor of the overlaid image is implied by the width and
|
||
|
|
height given in struct :c:type:`v4l2_window` and the size
|
||
|
|
of the cropping rectangle. For more information see :ref:`crop`.
|
||
|
|
|
||
|
|
When simultaneous capturing and overlay is supported and the hardware
|
||
|
|
prohibits different image and window sizes, the size requested first
|
||
|
|
takes precedence. The attempt to capture or overlay as well
|
||
|
|
(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error
|
||
|
|
code or return accordingly modified parameters.
|
||
|
|
|
||
|
|
|
||
|
|
.. c:type:: v4l2_window
|
||
|
|
|
||
|
|
struct v4l2_window
|
||
|
|
------------------
|
||
|
|
|
||
|
|
``struct v4l2_rect w``
|
||
|
|
Size and position of the window relative to the top, left corner of
|
||
|
|
the frame buffer defined with
|
||
|
|
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the
|
||
|
|
frame buffer width and height, the ``x`` and ``y`` coordinates can
|
||
|
|
be negative, and it can lie completely outside the frame buffer. The
|
||
|
|
driver clips the window accordingly, or if that is not possible,
|
||
|
|
modifies its size and/or position.
|
||
|
|
|
||
|
|
``enum v4l2_field field``
|
||
|
|
Applications set this field to determine which video field shall be
|
||
|
|
overlaid, typically one of ``V4L2_FIELD_ANY`` (0),
|
||
|
|
``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or
|
||
|
|
``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different
|
||
|
|
field order and return the actual setting here.
|
||
|
|
|
||
|
|
``__u32 chromakey``
|
||
|
|
When chroma-keying has been negotiated with
|
||
|
|
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field
|
||
|
|
to the desired pixel value for the chroma key. The format is the
|
||
|
|
same as the pixel format of the framebuffer (struct
|
||
|
|
:c:type:`v4l2_framebuffer` ``fmt.pixelformat``
|
||
|
|
field), with bytes in host order. E. g. for
|
||
|
|
:ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should
|
||
|
|
be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host.
|
||
|
|
|
||
|
|
``struct v4l2_clip * clips``
|
||
|
|
When chroma-keying has *not* been negotiated and
|
||
|
|
:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
|
||
|
|
applications can set this field to point to an array of clipping
|
||
|
|
rectangles.
|
||
|
|
|
||
|
|
Like the window coordinates w, clipping rectangles are defined
|
||
|
|
relative to the top, left corner of the frame buffer. However
|
||
|
|
clipping rectangles must not extend the frame buffer width and
|
||
|
|
height, and they must not overlap. If possible applications
|
||
|
|
should merge adjacent rectangles. Whether this must create
|
||
|
|
x-y or y-x bands, or the order of rectangles, is not defined. When
|
||
|
|
clip lists are not supported the driver ignores this field. Its
|
||
|
|
contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
|
||
|
|
are undefined.
|
||
|
|
|
||
|
|
``__u32 clipcount``
|
||
|
|
When the application set the ``clips`` field, this field must
|
||
|
|
contain the number of clipping rectangles in the list. When clip
|
||
|
|
lists are not supported the driver ignores this field, its contents
|
||
|
|
after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are
|
||
|
|
supported but no clipping is desired this field must be set to zero.
|
||
|
|
|
||
|
|
``void * bitmap``
|
||
|
|
When chroma-keying has *not* been negotiated and
|
||
|
|
:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
|
||
|
|
applications can set this field to point to a clipping bit mask.
|
||
|
|
|
||
|
|
It must be of the same size as the window, ``w.width`` and ``w.height``.
|
||
|
|
Each bit corresponds to a pixel in the overlaid image, which is
|
||
|
|
displayed only when the bit is *set*. Pixel coordinates translate to
|
||
|
|
bits like:
|
||
|
|
|
||
|
|
|
||
|
|
.. code-block:: c
|
||
|
|
|
||
|
|
((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7))
|
||
|
|
|
||
|
|
where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_
|
||
|
|
|
||
|
|
When a clipping bit mask is not supported the driver ignores this field,
|
||
|
|
its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are
|
||
|
|
undefined. When a bit mask is supported but no clipping is desired this
|
||
|
|
field must be set to ``NULL``.
|
||
|
|
|
||
|
|
Applications need not create a clip list or bit mask. When they pass
|
||
|
|
both, or despite negotiating chroma-keying, the results are undefined.
|
||
|
|
Regardless of the chosen method, the clipping abilities of the hardware
|
||
|
|
may be limited in quantity or quality. The results when these limits are
|
||
|
|
exceeded are undefined. [#f3]_
|
||
|
|
|
||
|
|
``__u8 global_alpha``
|
||
|
|
The global alpha value used to blend the framebuffer with video
|
||
|
|
images, if global alpha blending has been negotiated
|
||
|
|
(``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see
|
||
|
|
:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`,
|
||
|
|
:ref:`framebuffer-flags`).
|
||
|
|
|
||
|
|
.. note::
|
||
|
|
|
||
|
|
This field was added in Linux 2.6.23, extending the
|
||
|
|
structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>`
|
||
|
|
ioctls, which take a pointer to a :c:type:`v4l2_format`
|
||
|
|
parent structure with padding bytes at the end, are not affected.
|
||
|
|
|
||
|
|
|
||
|
|
.. c:type:: v4l2_clip
|
||
|
|
|
||
|
|
struct v4l2_clip [#f4]_
|
||
|
|
-----------------------
|
||
|
|
|
||
|
|
``struct v4l2_rect c``
|
||
|
|
Coordinates of the clipping rectangle, relative to the top, left
|
||
|
|
corner of the frame buffer. Only window pixels *outside* all
|
||
|
|
clipping rectangles are displayed.
|
||
|
|
|
||
|
|
``struct v4l2_clip * next``
|
||
|
|
Pointer to the next clipping rectangle, ``NULL`` when this is the last
|
||
|
|
rectangle. Drivers ignore this field, it cannot be used to pass a
|
||
|
|
linked list of clipping rectangles.
|
||
|
|
|
||
|
|
|
||
|
|
.. c:type:: v4l2_rect
|
||
|
|
|
||
|
|
struct v4l2_rect
|
||
|
|
----------------
|
||
|
|
|
||
|
|
``__s32 left``
|
||
|
|
Horizontal offset of the top, left corner of the rectangle, in
|
||
|
|
pixels.
|
||
|
|
|
||
|
|
``__s32 top``
|
||
|
|
Vertical offset of the top, left corner of the rectangle, in pixels.
|
||
|
|
Offsets increase to the right and down.
|
||
|
|
|
||
|
|
``__u32 width``
|
||
|
|
Width of the rectangle, in pixels.
|
||
|
|
|
||
|
|
``__u32 height``
|
||
|
|
Height of the rectangle, in pixels.
|
||
|
|
|
||
|
|
|
||
|
|
Enabling Overlay
|
||
|
|
================
|
||
|
|
|
||
|
|
To start or stop the frame buffer overlay applications call the
|
||
|
|
:ref:`VIDIOC_OVERLAY` ioctl.
|
||
|
|
|
||
|
|
.. [#f1]
|
||
|
|
In the opinion of the designers of this API, no driver writer taking
|
||
|
|
the efforts to support simultaneous capturing and overlay will
|
||
|
|
restrict this ability by requiring a single file descriptor, as in
|
||
|
|
V4L and earlier versions of V4L2. Making this optional means
|
||
|
|
applications depending on two file descriptors need backup routines
|
||
|
|
to be compatible with all drivers, which is considerable more work
|
||
|
|
than using two fds in applications which do not. Also two fd's fit
|
||
|
|
the general concept of one file descriptor for each logical stream.
|
||
|
|
Hence as a complexity trade-off drivers *must* support two file
|
||
|
|
descriptors and *may* support single fd operation.
|
||
|
|
|
||
|
|
.. [#f2]
|
||
|
|
Should we require ``w.width`` to be a multiple of eight?
|
||
|
|
|
||
|
|
.. [#f3]
|
||
|
|
When the image is written into frame buffer memory it will be
|
||
|
|
undesirable if the driver clips out less pixels than expected,
|
||
|
|
because the application and graphics system are not aware these
|
||
|
|
regions need to be refreshed. The driver should clip out more pixels
|
||
|
|
or not write the image at all.
|
||
|
|
|
||
|
|
.. [#f4]
|
||
|
|
The X Window system defines "regions" which are vectors of ``struct
|
||
|
|
BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and
|
||
|
|
``height = y2 - y1``, so one cannot pass X11 clip lists directly.
|