1.. SPDX-License-Identifier: GPL-2.0
2
3Writing camera sensor drivers
4=============================
5
6CSI-2 and parallel (BT.601 and BT.656) busses
7---------------------------------------------
8
9Please see :ref:`transmitter-receiver`.
10
11Handling clocks
12---------------
13
14Camera sensors have an internal clock tree including a PLL and a number of
15divisors. The clock tree is generally configured by the driver based on a few
16input parameters that are specific to the hardware:: the external clock frequency
17and the link frequency. The two parameters generally are obtained from system
18firmware. **No other frequencies should be used in any circumstances.**
19
20The reason why the clock frequencies are so important is that the clock signals
21come out of the SoC, and in many cases a specific frequency is designed to be
22used in the system. Using another frequency may cause harmful effects
23elsewhere. Therefore only the pre-determined frequencies are configurable by the
24user.
25
26ACPI
27~~~~
28
29Read the ``clock-frequency`` _DSD property to denote the frequency. The driver
30can rely on this frequency being used.
31
32Devicetree
33~~~~~~~~~~
34
35The currently preferred way to achieve this is using ``assigned-clocks``,
36``assigned-clock-parents`` and ``assigned-clock-rates`` properties. See
37``Documentation/devicetree/bindings/clock/clock-bindings.txt`` for more
38information. The driver then gets the frequency using ``clk_get_rate()``.
39
40This approach has the drawback that there's no guarantee that the frequency
41hasn't been modified directly or indirectly by another driver, or supported by
42the board's clock tree to begin with. Changes to the Common Clock Framework API
43are required to ensure reliability.
44
45Frame size
46----------
47
48There are two distinct ways to configure the frame size produced by camera
49sensors.
50
51Freely configurable camera sensor drivers
52~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
53
54Freely configurable camera sensor drivers expose the device's internal
55processing pipeline as one or more sub-devices with different cropping and
56scaling configurations. The output size of the device is the result of a series
57of cropping and scaling operations from the device's pixel array's size.
58
59An example of such a driver is the CCS driver (see ``drivers/media/i2c/ccs``).
60
61Register list based drivers
62~~~~~~~~~~~~~~~~~~~~~~~~~~~
63
64Register list based drivers generally, instead of able to configure the device
65they control based on user requests, are limited to a number of preset
66configurations that combine a number of different parameters that on hardware
67level are independent. How a driver picks such configuration is based on the
68format set on a source pad at the end of the device's internal pipeline.
69
70Most sensor drivers are implemented this way, see e.g.
71``drivers/media/i2c/imx319.c`` for an example.
72
73Frame interval configuration
74----------------------------
75
76There are two different methods for obtaining possibilities for different frame
77intervals as well as configuring the frame interval. Which one to implement
78depends on the type of the device.
79
80Raw camera sensors
81~~~~~~~~~~~~~~~~~~
82
83Instead of a high level parameter such as frame interval, the frame interval is
84a result of the configuration of a number of camera sensor implementation
85specific parameters. Luckily, these parameters tend to be the same for more or
86less all modern raw camera sensors.
87
88The frame interval is calculated using the following equation::
89
90	frame interval = (analogue crop width + horizontal blanking) *
91			 (analogue crop height + vertical blanking) / pixel rate
92
93The formula is bus independent and is applicable for raw timing parameters on
94large variety of devices beyond camera sensors. Devices that have no analogue
95crop, use the full source image size, i.e. pixel array size.
96
97Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
98``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
99is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
100the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
101sub-device. The unit of that control is pixels per second.
102
103Register list based drivers need to implement read-only sub-device nodes for the
104purpose. Devices that are not register list based need these to configure the
105device's internal processing pipeline.
106
107The first entity in the linear pipeline is the pixel array. The pixel array may
108be followed by other entities that are there to allow configuring binning,
109skipping, scaling or digital crop :ref:`v4l2-subdev-selections`.
110
111USB cameras etc. devices
112~~~~~~~~~~~~~~~~~~~~~~~~
113
114USB video class hardware, as well as many cameras offering a similar higher
115level interface natively, generally use the concept of frame interval (or frame
116rate) on device level in firmware or hardware. This means lower level controls
117implemented by raw cameras may not be used on uAPI (or even kAPI) to control the
118frame interval on these devices.
119
120Power management
121----------------
122
123Always use runtime PM to manage the power states of your device. Camera sensor
124drivers are in no way special in this respect: they are responsible for
125controlling the power state of the device they otherwise control as well. In
126general, the device must be powered on at least when its registers are being
127accessed and when it is streaming.
128
129Existing camera sensor drivers may rely on the old
130struct v4l2_subdev_core_ops->s_power() callback for bridge or ISP drivers to
131manage their power state. This is however **deprecated**. If you feel you need
132to begin calling an s_power from an ISP or a bridge driver, instead please add
133runtime PM support to the sensor driver you are using. Likewise, new drivers
134should not use s_power.
135
136Please see examples in e.g. ``drivers/media/i2c/ov8856.c`` and
137``drivers/media/i2c/ccs/ccs-core.c``. The two drivers work in both ACPI
138and DT based systems.
139
140Control framework
141~~~~~~~~~~~~~~~~~
142
143``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime
144PM ``runtime_resume`` callback, as it has no way to figure out the power state
145of the device. This is because the power state of the device is only changed
146after the power state transition has taken place. The ``s_ctrl`` callback can be
147used to obtain device's power state after the power state transition:
148
149.. c:function:: int pm_runtime_get_if_in_use(struct device *dev);
150
151The function returns a non-zero value if it succeeded getting the power count or
152runtime PM was disabled, in either of which cases the driver may proceed to
153access the device.
154
155Rotation, orientation and flipping
156----------------------------------
157
158Some systems have the camera sensor mounted upside down compared to its natural
159mounting rotation. In such cases, drivers shall expose the information to
160userspace with the :ref:`V4L2_CID_CAMERA_SENSOR_ROTATION
161<v4l2-camera-sensor-rotation>` control.
162
163Sensor drivers shall also report the sensor's mounting orientation with the
164:ref:`V4L2_CID_CAMERA_SENSOR_ORIENTATION <v4l2-camera-sensor-orientation>`.
165
166Use ``v4l2_fwnode_device_parse()`` to obtain rotation and orientation
167information from system firmware and ``v4l2_ctrl_new_fwnode_properties()`` to
168register the appropriate controls.
169
170Sensor drivers that have any vertical or horizontal flips embedded in the
171register programming sequences shall initialize the V4L2_CID_HFLIP and
172V4L2_CID_VFLIP controls with the values programmed by the register sequences.
173The default values of these controls shall be 0 (disabled). Especially these
174controls shall not be inverted, independently of the sensor's mounting
175rotation.
176