1.. SPDX-License-Identifier: GPL-2.0 2 3Writing camera sensor drivers 4============================= 5 6CSI-2 7----- 8 9Please see what is written on :ref:`MIPI_CSI_2`. 10 11Handling clocks 12--------------- 13 14Camera sensors have an internal clock tree including a PLL and a number of 15divisors. The clock tree is generally configured by the driver based on a few 16input parameters that are specific to the hardware:: the external clock frequency 17and the link frequency. The two parameters generally are obtained from system 18firmware. No other frequencies should be used in any circumstances. 19 20The reason why the clock frequencies are so important is that the clock signals 21come out of the SoC, and in many cases a specific frequency is designed to be 22used in the system. Using another frequency may cause harmful effects 23elsewhere. Therefore only the pre-determined frequencies are configurable by the 24user. 25 26Frame size 27---------- 28 29There are two distinct ways to configure the frame size produced by camera 30sensors. 31 32Freely configurable camera sensor drivers 33~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 34 35Freely configurable camera sensor drivers expose the device's internal 36processing pipeline as one or more sub-devices with different cropping and 37scaling configurations. The output size of the device is the result of a series 38of cropping and scaling operations from the device's pixel array's size. 39 40An example of such a driver is the smiapp driver (see drivers/media/i2c/smiapp). 41 42Register list based drivers 43~~~~~~~~~~~~~~~~~~~~~~~~~~~ 44 45Register list based drivers generally, instead of able to configure the device 46they control based on user requests, are limited to a number of preset 47configurations that combine a number of different parameters that on hardware 48level are independent. How a driver picks such configuration is based on the 49format set on a source pad at the end of the device's internal pipeline. 50 51Most sensor drivers are implemented this way, see e.g. 52drivers/media/i2c/imx319.c for an example. 53 54Frame interval configuration 55---------------------------- 56 57There are two different methods for obtaining possibilities for different frame 58intervals as well as configuring the frame interval. Which one to implement 59depends on the type of the device. 60 61Raw camera sensors 62~~~~~~~~~~~~~~~~~~ 63 64Instead of a high level parameter such as frame interval, the frame interval is 65a result of the configuration of a number of camera sensor implementation 66specific parameters. Luckily, these parameters tend to be the same for more or 67less all modern raw camera sensors. 68 69The frame interval is calculated using the following equation:: 70 71 frame interval = (analogue crop width + horizontal blanking) * 72 (analogue crop height + vertical blanking) / pixel rate 73 74The formula is bus independent and is applicable for raw timing parameters on 75large variety of devices beyond camera sensors. Devices that have no analogue 76crop, use the full source image size, i.e. pixel array size. 77 78Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and 79``V4L2_CID_VBLANK``, respectively. The unit of these controls are lines. The 80pixel rate is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The 81unit of that control is Hz. 82 83Register list based drivers need to implement read-only sub-device nodes for the 84purpose. Devices that are not register list based need these to configure the 85device's internal processing pipeline. 86 87The first entity in the linear pipeline is the pixel array. The pixel array may 88be followed by other entities that are there to allow configuring binning, 89skipping, scaling or digital crop :ref:`v4l2-subdev-selections`. 90 91USB cameras etc. devices 92~~~~~~~~~~~~~~~~~~~~~~~~ 93 94USB video class hardware, as well as many cameras offering a similar higher 95level interface natively, generally use the concept of frame interval (or frame 96rate) on device level in firmware or hardware. This means lower level controls 97implemented by raw cameras may not be used on uAPI (or even kAPI) to control the 98frame interval on these devices. 99 100Power management 101---------------- 102 103Always use runtime PM to manage the power states of your device. Camera sensor 104drivers are in no way special in this respect: they are responsible for 105controlling the power state of the device they otherwise control as well. In 106general, the device must be powered on at least when its registers are being 107accessed and when it is streaming. 108 109Existing camera sensor drivers may rely on the old 110:c:type:`v4l2_subdev_core_ops`->s_power() callback for bridge or ISP drivers to 111manage their power state. This is however **deprecated**. If you feel you need 112to begin calling an s_power from an ISP or a bridge driver, instead please add 113runtime PM support to the sensor driver you are using. Likewise, new drivers 114should not use s_power. 115 116Please see examples in e.g. ``drivers/media/i2c/ov8856.c`` and 117``drivers/media/i2c/smiapp/smiapp-core.c``. The two drivers work in both ACPI 118and DT based systems. 119 120Control framework 121~~~~~~~~~~~~~~~~~ 122 123``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime 124PM ``runtime_resume`` callback, as it has no way to figure out the power state 125of the device. This is because the power state of the device is only changed 126after the power state transition has taken place. The ``s_ctrl`` callback can be 127used to obtain device's power state after the power state transition: 128 129.. c:function:: 130 int pm_runtime_get_if_in_use(struct device *dev); 131 132The function returns a non-zero value if it succeeded getting the power count or 133runtime PM was disabled, in either of which cases the driver may proceed to 134access the device. 135 136Controls 137-------- 138 139For camera sensors that are connected to a bus where transmitter and receiver 140require common configuration set by drivers, such as CSI-2 or parallel (BT.601 141or BT.656) bus, the ``V4L2_CID_LINK_FREQ`` control is mandatory on transmitter 142drivers. Receiver drivers can use the ``V4L2_CID_LINK_FREQ`` to query the 143frequency used on the bus. 144 145The transmitter drivers should also implement ``V4L2_CID_PIXEL_RATE`` control in 146order to tell the maximum pixel rate to the receiver. This is required on raw 147camera sensors. 148