1.. SPDX-License-Identifier: GPL-2.0
2
3Writing camera sensor drivers
4=============================
5
6CSI-2
7-----
8
9Please see what is written on :ref:`MIPI_CSI_2`.
10
11Handling clocks
12---------------
13
14Camera sensors have an internal clock tree including a PLL and a number of
15divisors. The clock tree is generally configured by the driver based on a few
16input parameters that are specific to the hardware:: the external clock frequency
17and the link frequency. The two parameters generally are obtained from system
18firmware. No other frequencies should be used in any circumstances.
19
20The reason why the clock frequencies are so important is that the clock signals
21come out of the SoC, and in many cases a specific frequency is designed to be
22used in the system. Using another frequency may cause harmful effects
23elsewhere. Therefore only the pre-determined frequencies are configurable by the
24user.
25
26Frame size
27----------
28
29There are two distinct ways to configure the frame size produced by camera
30sensors.
31
32Freely configurable camera sensor drivers
33~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
34
35Freely configurable camera sensor drivers expose the device's internal
36processing pipeline as one or more sub-devices with different cropping and
37scaling configurations. The output size of the device is the result of a series
38of cropping and scaling operations from the device's pixel array's size.
39
40An example of such a driver is the smiapp driver (see drivers/media/i2c/smiapp).
41
42Register list based drivers
43~~~~~~~~~~~~~~~~~~~~~~~~~~~
44
45Register list based drivers generally, instead of able to configure the device
46they control based on user requests, are limited to a number of preset
47configurations that combine a number of different parameters that on hardware
48level are independent. How a driver picks such configuration is based on the
49format set on a source pad at the end of the device's internal pipeline.
50
51Most sensor drivers are implemented this way, see e.g.
52drivers/media/i2c/imx319.c for an example.
53
54Frame interval configuration
55----------------------------
56
57There are two different methods for obtaining possibilities for different frame
58intervals as well as configuring the frame interval. Which one to implement
59depends on the type of the device.
60
61Raw camera sensors
62~~~~~~~~~~~~~~~~~~
63
64Instead of a high level parameter such as frame interval, the frame interval is
65a result of the configuration of a number of camera sensor implementation
66specific parameters. Luckily, these parameters tend to be the same for more or
67less all modern raw camera sensors.
68
69The frame interval is calculated using the following equation::
70
71	frame interval = (analogue crop width + horizontal blanking) *
72			 (analogue crop height + vertical blanking) / pixel rate
73
74The formula is bus independent and is applicable for raw timing parameters on
75large variety of devices beyond camera sensors. Devices that have no analogue
76crop, use the full source image size, i.e. pixel array size.
77
78Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
79``V4L2_CID_VBLANK``, respectively. The unit of these controls are lines. The
80pixel rate is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The
81unit of that control is Hz.
82
83Register list based drivers need to implement read-only sub-device nodes for the
84purpose. Devices that are not register list based need these to configure the
85device's internal processing pipeline.
86
87The first entity in the linear pipeline is the pixel array. The pixel array may
88be followed by other entities that are there to allow configuring binning,
89skipping, scaling or digital crop :ref:`v4l2-subdev-selections`.
90
91USB cameras etc. devices
92~~~~~~~~~~~~~~~~~~~~~~~~
93
94USB video class hardware, as well as many cameras offering a similar higher
95level interface natively, generally use the concept of frame interval (or frame
96rate) on device level in firmware or hardware. This means lower level controls
97implemented by raw cameras may not be used on uAPI (or even kAPI) to control the
98frame interval on these devices.
99
100Power management
101----------------
102
103Always use runtime PM to manage the power states of your device. Camera sensor
104drivers are in no way special in this respect: they are responsible for
105controlling the power state of the device they otherwise control as well. In
106general, the device must be powered on at least when its registers are being
107accessed and when it is streaming.
108
109Existing camera sensor drivers may rely on the old
110:c:type:`v4l2_subdev_core_ops`->s_power() callback for bridge or ISP drivers to
111manage their power state. This is however **deprecated**. If you feel you need
112to begin calling an s_power from an ISP or a bridge driver, instead please add
113runtime PM support to the sensor driver you are using. Likewise, new drivers
114should not use s_power.
115
116Please see examples in e.g. ``drivers/media/i2c/ov8856.c`` and
117``drivers/media/i2c/smiapp/smiapp-core.c``. The two drivers work in both ACPI
118and DT based systems.
119
120Control framework
121~~~~~~~~~~~~~~~~~
122
123``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime
124PM ``runtime_resume`` callback, as it has no way to figure out the power state
125of the device. This is because the power state of the device is only changed
126after the power state transition has taken place. The ``s_ctrl`` callback can be
127used to obtain device's power state after the power state transition:
128
129.. c:function::
130	int pm_runtime_get_if_in_use(struct device *dev);
131
132The function returns a non-zero value if it succeeded getting the power count or
133runtime PM was disabled, in either of which cases the driver may proceed to
134access the device.
135