Lines Matching +full:ref +full:- +full:pad
1 .. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
6 Sub-device Interface
13 components as software blocks called sub-devices.
15 V4L2 sub-devices are usually kernel-only objects. If the V4L2 driver
17 media entities. Applications will be able to enumerate the sub-devices
21 In addition to make sub-devices discoverable, drivers can also choose to
23 sub-device driver and the V4L2 device driver support this, sub-devices
26 - query, read and write sub-devices controls
28 - subscribe and unsubscribe to events and retrieve them
30 - negotiate image formats on individual pads
32 - inspect and modify internal data routing between pads of the same entity
34 Sub-device character device nodes, conventionally named
35 ``/dev/v4l-subdev*``, use major number 81.
37 Drivers may opt to limit the sub-device character devices to only expose
38 operations that do not modify the device state. In such a case the sub-devices
39 are referred to as ``read-only`` in the rest of this documentation, and the
46 Most V4L2 controls are implemented by sub-device hardware. Drivers
48 Applications can control all sub-devices through a single interface.
57 Applications can access those hidden controls through the sub-device
58 node with the V4L2 control API described in :ref:`control`. The ioctls
61 sub-device.
70 V4L2 sub-devices can notify applications of events as described in
71 :ref:`event`. The API behaves identically as when used on V4L2 device
73 the sub-device. Depending on the driver, those events might also be
77 .. _pad-level-formats:
79 Pad-level Formats
84 Pad-level formats are only applicable to very complex devices that
85 need to expose low-level format configuration to user space. Generic
96 :ref:`selection <VIDIOC_SUBDEV_G_SELECTION>` ioctls. The driver is
103 :ref:`pipeline-scaling`, where image scaling can be performed on both
107 .. _pipeline-scaling:
109 .. kernel-figure:: pipeline.dot
125 Drivers that implement the :ref:`media API <media-controller-intro>`
126 can expose pad-level image format configuration to applications. When
128 :ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` and
129 :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls. to
130 negotiate formats on a per-pad basis.
135 :ref:`VIDIOC_STREAMON <VIDIOC_STREAMON>` time, and an ``EPIPE`` error
138 Pad-level image format configuration support can be tested by calling
139 the :ref:`VIDIOC_SUBDEV_G_FMT` ioctl on pad
140 0. If the driver returns an ``EINVAL`` error code pad-level format
141 configuration is not supported by the sub-device.
145 ------------------
155 :ref:`V4L2_SUBDEV_FORMAT_TRY <VIDIOC_SUBDEV_G_FMT>`, the
156 :ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` and
157 :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls operate on
164 the sub-device file handles. A
165 :ref:`VIDIOC_SUBDEV_G_FMT <VIDIOC_SUBDEV_G_FMT>` call will return
166 the last try format set *on the same sub-device file handle*. Several
167 applications querying the same sub-device at the same time will thus not
172 :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctl. Drivers
181 to an :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` call as-is
185 Drivers automatically propagate formats inside sub-devices. When a try
186 or active format is set on a pad, corresponding formats on other pads of
187 the same sub-device can be modified by the driver. Drivers are free to
191 - Formats should be propagated from sink pads to source pads. Modifying
192 a format on a source pad should not modify the format on any sink
193 pad.
195 - Sub-devices that scale frames using variable scaling factors should
201 propagating them from one sub-device file handle to another.
207 :ref:`sample-pipeline-config` shows a sample configuration sequence
208 for the pipeline described in :ref:`pipeline-scaling` (table columns
209 list entity names and pad numbers).
220 .. _sample-pipeline-config:
222 .. flat-table:: Sample Pipeline Configuration
223 :header-rows: 1
224 :stub-columns: 0
227 * -
228 - Sensor/0
231 - Frontend/0
234 - Frontend/1
237 - Scaler/0
240 - Scaler/0
243 - Scaler/1
246 * - Initial state
247 - 2048x1536
250 - (default)
251 - (default)
252 - (default)
253 - (default)
254 - (default)
255 * - Configure frontend sink format
256 - 2048x1536
259 - *2048x1536*
262 - *2046x1534*
265 - (default)
266 - (default)
267 - (default)
268 * - Configure scaler sink format
269 - 2048x1536
272 - 2048x1536
275 - 2046x1534
278 - *2046x1534*
281 - *0,0/2046x1534*
282 - *2046x1534*
285 * - Configure scaler sink compose selection
286 - 2048x1536
289 - 2048x1536
292 - 2046x1534
295 - 2046x1534
298 - *0,0/1280x960*
299 - *1280x960*
307 1. Initial state. The sensor source pad format is set to its native 3MP
310 values, as well as the compose rectangle on the scaler's sink pad.
312 2. The application configures the frontend sink pad format's size to
314 driver propagates the format to the frontend source pad.
316 3. The application configures the scaler sink pad format's size to
319 on the sink pad is set to V4L2_MBUS_FMT_SGRBG_1X8. The driver
321 scaler's sink pad, and the format to the scaler source pad.
324 rectangle of the scaler's sink pad 1280x960. The driver propagates
325 the size to the scaler's source pad format.
334 be applied as-is by the driver without being modified.
337 .. _v4l2-subdev-selections:
340 ---------------------------------------------
342 Many sub-devices support cropping frames on their input or output pads
353 As for pad formats, drivers store try and active rectangles for the
354 selection targets :ref:`v4l2-selections-common`.
356 On sink pads, cropping is applied relative to the current pad format.
357 The pad format represents the image size as received by the sub-device
359 represents the sub-image that will be transmitted further inside the
360 sub-device for processing.
368 rectangle on the subdev's sink pad is scaled to the size configured
370 :ref:`VIDIOC_SUBDEV_S_SELECTION <VIDIOC_SUBDEV_G_SELECTION>` IOCTL
371 using ``V4L2_SEL_TGT_COMPOSE`` selection target on the same pad. If the
377 COMPOSE rectangle on the sink pad. In both sink and source pads, the
384 the image size either up or down. :ref:`v4l2-selection-flags`
388 --------------------------
405 pixel array is not rectangular but cross-shaped or round. The maximum
409 .. _format-propagation:
412 ---------------------------------------------
415 the sink pad towards the source pad. This is also reflected in the order
425 rectangle, which refers to the sink compose bounds rectangle --- if it
428 1. Sink pad format. The user configures the sink pad format. This format
430 pad for further processing.
432 2. Sink pad actual crop selection. The sink pad crop defines the crop
433 performed to the sink pad format.
435 3. Sink pad actual compose selection. The size of the sink pad compose
437 pad crop rectangle. The location of the compose rectangle specifies
441 4. Source pad actual crop selection. Crop on the source pad defines crop
444 5. Source pad format. The source pad format defines the output pixel
447 by the size of the source pad actual crop selection.
453 selection will refer to the sink pad format dimensions instead.
456 .. _subdev-image-processing-crop:
458 .. kernel-figure:: subdev-image-processing-crop.svg
459 :alt: subdev-image-processing-crop.svg
464 In the above example, the subdev supports cropping on its sink pad. To
466 pad. Now the actual crop rectangle can be set on the sink pad --- the
470 pad.
473 .. _subdev-image-processing-scaling-multi-source:
475 .. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
476 :alt: subdev-image-processing-scaling-multi-source.svg
489 .. _subdev-image-processing-full:
491 .. kernel-figure:: subdev-image-processing-full.svg
492 :alt: subdev-image-processing-full.svg
507 subdev-formats
510 ----------------------------------------------------
512 Simple V4L2 sub-devices do not support multiple, unrelated video streams,
513 and only a single stream can pass through a media link and a media pad.
514 Thus each pad contains a format and selection configuration for that
517 subdev are still a single stream per pad.
519 Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple
521 link connecting a transmitter source pad with a sink pad on the receiver. For
524 by a media link which connects the single sensor's source pad with the receiver
525 sink pad. The stream-aware receiver will de-multiplex the streams received on
526 the its sink pad and allows to route them individually to one of its source
530 non-multiplexed subdev drivers, but, of course, require a routing configuration
540 streams from one end of the link to the other, and sub-devices have routing
544 A stream ID is a media pad-local identifier for a stream. Streams IDs of
548 of the sub-device.
551 sub-device and a (pad, stream) pair. For sub-devices that do not support
557 The addition of streams to the V4L2 sub-device interface moves the sub-device
558 formats and selections from pads to (pad, stream) pairs. Besides the
559 usual pad, also the stream ID needs to be provided for setting formats and
561 the same as without streams (see :ref:`format-propagation`).
563 Instead of the sub-device wide merging of streams from all sink pads
567 stream on a source pad, however, only a single route is allowed.
569 Any configurations of a stream within a pad, such as format or selections,
576 The configuration of the streams is done individually for each sub-device and
577 the validity of the streams between sub-devices is validated when the pipeline
582 1) Set up links. Connect the pads between sub-devices using the :ref:`Media
586 setting the routing table for the sub-device using
587 :ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that
589 sub-device to default values.
592 are configured separately as documented for plain sub-devices in
593 :ref:`format-propagation`. The stream ID is set to the same stream ID
595 :ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl.
602 - Two identical sensors (Sensor A and Sensor B). Each sensor has a single source
603 pad (pad 0) which carries a pixel data stream.
605 - Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the
606 sensors (pads 0, 1), and one source pad (pad 2), which outputs two streams.
608 - Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0),
609 connected to the bridge, and two source pads (pads 1-2), going to the DMA
612 - DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is
613 connected to a single source pad in the receiver.
615 The sensors, the bridge and the receiver are modeled as V4L2 sub-devices,
616 exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
623 not differ from normal non-multiplexed media controller setup.
627 .. flat-table:: Bridge routing table
628 :header-rows: 1
630 * - Sink Pad/Stream
631 - Source Pad/Stream
632 - Routing Flags
633 - Comments
634 * - 0/0
635 - 2/0
636 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
637 - Pixel data stream from Sensor A
638 * - 1/0
639 - 2/1
640 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
641 - Pixel data stream from Sensor B
643 .. flat-table:: Receiver routing table
644 :header-rows: 1
646 * - Sink Pad/Stream
647 - Source Pad/Stream
648 - Routing Flags
649 - Comments
650 * - 0/0
651 - 1/0
652 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
653 - Pixel data stream from Sensor A
654 * - 0/1
655 - 2/0
656 - V4L2_SUBDEV_ROUTE_FL_ACTIVE
657 - Pixel data stream from Sensor B
668 using :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
669 stream endpoint in each sub-device.