1Common bindings for video receiver and transmitter interfaces 2 3General concept 4--------------- 5 6Video data pipelines usually consist of external devices, e.g. camera sensors, 7controlled over an I2C, SPI or UART bus, and SoC internal IP blocks, including 8video DMA engines and video data processors. 9 10SoC internal blocks are described by DT nodes, placed similarly to other SoC 11blocks. External devices are represented as child nodes of their respective 12bus controller nodes, e.g. I2C. 13 14Data interfaces on all video devices are described by their child 'port' nodes. 15Configuration of a port depends on other devices participating in the data 16transfer and is described by 'endpoint' subnodes. 17 18device { 19 ... 20 ports { 21 #address-cells = <1>; 22 #size-cells = <0>; 23 24 port@0 { 25 ... 26 endpoint@0 { ... }; 27 endpoint@1 { ... }; 28 }; 29 port@1 { ... }; 30 }; 31}; 32 33If a port can be configured to work with more than one remote device on the same 34bus, an 'endpoint' child node must be provided for each of them. If more than 35one port is present in a device node or there is more than one endpoint at a 36port, or port node needs to be associated with a selected hardware interface, 37a common scheme using '#address-cells', '#size-cells' and 'reg' properties is 38used. 39 40All 'port' nodes can be grouped under optional 'ports' node, which allows to 41specify #address-cells, #size-cells properties independently for the 'port' 42and 'endpoint' nodes and any child device nodes a device might have. 43 44Two 'endpoint' nodes are linked with each other through their 'remote-endpoint' 45phandles. An endpoint subnode of a device contains all properties needed for 46configuration of this device for data exchange with other device. In most 47cases properties at the peer 'endpoint' nodes will be identical, however they 48might need to be different when there is any signal modifications on the bus 49between two devices, e.g. there are logic signal inverters on the lines. 50 51It is allowed for multiple endpoints at a port to be active simultaneously, 52where supported by a device. For example, in case where a data interface of 53a device is partitioned into multiple data busses, e.g. 16-bit input port 54divided into two separate ITU-R BT.656 8-bit busses. In such case bus-width 55and data-shift properties can be used to assign physical data lines to each 56endpoint node (logical bus). 57 58Documenting bindings for devices 59-------------------------------- 60 61All required and optional bindings the device supports shall be explicitly 62documented in device DT binding documentation. This also includes port and 63endpoint nodes for the device, including unit-addresses and reg properties where 64relevant. 65 66Please also see Documentation/devicetree/bindings/graph.txt . 67 68Required properties 69------------------- 70 71If there is more than one 'port' or more than one 'endpoint' node or 'reg' 72property is present in port and/or endpoint nodes the following properties 73are required in a relevant parent node: 74 75 - #address-cells : number of cells required to define port/endpoint 76 identifier, should be 1. 77 - #size-cells : should be zero. 78 79 80Optional properties 81------------------- 82 83- flash-leds: An array of phandles, each referring to a flash LED, a sub-node 84 of the LED driver device node. 85 86- lens-focus: A phandle to the node of the focus lens controller. 87 88- rotation: The camera rotation is expressed as the angular difference in 89 degrees between two reference systems, one relative to the camera module, and 90 one defined on the external world scene to be captured when projected on the 91 image sensor pixel array. 92 93 A camera sensor has a 2-dimensional reference system 'Rc' defined by 94 its pixel array read-out order. The origin is set to the first pixel 95 being read out, the X-axis points along the column read-out direction 96 towards the last columns, and the Y-axis along the row read-out 97 direction towards the last row. 98 99 A typical example for a sensor with a 2592x1944 pixel array matrix 100 observed from the front is: 101 102 2591 X-axis 0 103 <------------------------+ 0 104 .......... ... ..........! 105 .......... ... ..........! Y-axis 106 ... ! 107 .......... ... ..........! 108 .......... ... ..........! 1943 109 V 110 111 The external world scene reference system 'Rs' is a 2-dimensional 112 reference system on the focal plane of the camera module. The origin is 113 placed on the top-left corner of the visible scene, the X-axis points 114 towards the right, and the Y-axis points towards the bottom of the 115 scene. The top, bottom, left and right directions are intentionally not 116 defined and depend on the environment in which the camera is used. 117 118 A typical example of a (very common) picture of a shark swimming from 119 left to right, as seen from the camera, is: 120 121 0 X-axis 122 0 +-------------------------------------> 123 ! 124 ! 125 ! 126 ! |\____)\___ 127 ! ) _____ __`< 128 ! |/ )/ 129 ! 130 ! 131 ! 132 V 133 Y-axis 134 135 with the reference system 'Rs' placed on the camera focal plane: 136 137 ¸.·˙! 138 ¸.·˙ ! 139 _ ¸.·˙ ! 140 +-/ \-+¸.·˙ ! 141 | (o) | ! Camera focal plane 142 +-----+˙·.¸ ! 143 ˙·.¸ ! 144 ˙·.¸ ! 145 ˙·.¸! 146 147 When projected on the sensor's pixel array, the image and the associated 148 reference system 'Rs' are typically (but not always) inverted, due to 149 the camera module's lens optical inversion effect. 150 151 Assuming the above represented scene of the swimming shark, the lens 152 inversion projects the scene and its reference system onto the sensor 153 pixel array, seen from the front of the camera sensor, as follows: 154 155 Y-axis 156 ^ 157 ! 158 ! 159 ! 160 ! |\_____)\__ 161 ! ) ____ ___.< 162 ! |/ )/ 163 ! 164 ! 165 ! 166 0 +-------------------------------------> 167 0 X-axis 168 169 Note the shark being upside-down. 170 171 The resulting projected reference system is named 'Rp'. 172 173 The camera rotation property is then defined as the angular difference 174 in the counter-clockwise direction between the camera reference system 175 'Rc' and the projected scene reference system 'Rp'. It is expressed in 176 degrees as a number in the range [0, 360[. 177 178 Examples 179 180 0 degrees camera rotation: 181 182 183 Y-Rp 184 ^ 185 Y-Rc ! 186 ^ ! 187 ! ! 188 ! ! 189 ! ! 190 ! ! 191 ! ! 192 ! ! 193 ! ! 194 ! 0 +-------------------------------------> 195 ! 0 X-Rp 196 0 +-------------------------------------> 197 0 X-Rc 198 199 200 X-Rc 0 201 <------------------------------------+ 0 202 X-Rp 0 ! 203 <------------------------------------+ 0 ! 204 ! ! 205 ! ! 206 ! ! 207 ! ! 208 ! ! 209 ! ! 210 ! ! 211 ! V 212 ! Y-Rc 213 V 214 Y-Rp 215 216 90 degrees camera rotation: 217 218 0 Y-Rc 219 0 +--------------------> 220 ! Y-Rp 221 ! ^ 222 ! ! 223 ! ! 224 ! ! 225 ! ! 226 ! ! 227 ! ! 228 ! ! 229 ! ! 230 ! ! 231 ! 0 +-------------------------------------> 232 ! 0 X-Rp 233 ! 234 ! 235 ! 236 ! 237 V 238 X-Rc 239 240 180 degrees camera rotation: 241 242 0 243 <------------------------------------+ 0 244 X-Rc ! 245 Y-Rp ! 246 ^ ! 247 ! ! 248 ! ! 249 ! ! 250 ! ! 251 ! ! 252 ! ! 253 ! V 254 ! Y-Rc 255 0 +-------------------------------------> 256 0 X-Rp 257 258 270 degrees camera rotation: 259 260 0 Y-Rc 261 0 +--------------------> 262 ! 0 263 ! <-----------------------------------+ 0 264 ! X-Rp ! 265 ! ! 266 ! ! 267 ! ! 268 ! ! 269 ! ! 270 ! ! 271 ! ! 272 ! ! 273 ! V 274 ! Y-Rp 275 ! 276 ! 277 ! 278 ! 279 V 280 X-Rc 281 282 283 Example one - Webcam 284 285 A camera module installed on the user facing part of a laptop screen 286 casing used for video calls. The captured images are meant to be 287 displayed in landscape mode (width > height) on the laptop screen. 288 289 The camera is typically mounted upside-down to compensate the lens 290 optical inversion effect: 291 292 Y-Rp 293 Y-Rc ^ 294 ^ ! 295 ! ! 296 ! ! |\_____)\__ 297 ! ! ) ____ ___.< 298 ! ! |/ )/ 299 ! ! 300 ! ! 301 ! ! 302 ! 0 +-------------------------------------> 303 ! 0 X-Rp 304 0 +-------------------------------------> 305 0 X-Rc 306 307 The two reference systems are aligned, the resulting camera rotation is 308 0 degrees, no rotation correction needs to be applied to the resulting 309 image once captured to memory buffers to correctly display it to users: 310 311 +--------------------------------------+ 312 ! ! 313 ! ! 314 ! ! 315 ! |\____)\___ ! 316 ! ) _____ __`< ! 317 ! |/ )/ ! 318 ! ! 319 ! ! 320 ! ! 321 +--------------------------------------+ 322 323 If the camera sensor is not mounted upside-down to compensate for the 324 lens optical inversion, the two reference systems will not be aligned, 325 with 'Rp' being rotated 180 degrees relatively to 'Rc': 326 327 328 X-Rc 0 329 <------------------------------------+ 0 330 ! 331 Y-Rp ! 332 ^ ! 333 ! ! 334 ! |\_____)\__ ! 335 ! ) ____ ___.< ! 336 ! |/ )/ ! 337 ! ! 338 ! ! 339 ! V 340 ! Y-Rc 341 0 +-------------------------------------> 342 0 X-Rp 343 344 The image once captured to memory will then be rotated by 180 degrees: 345 346 +--------------------------------------+ 347 ! ! 348 ! ! 349 ! ! 350 ! __/(_____/| ! 351 ! >.___ ____ ( ! 352 ! \( \| ! 353 ! ! 354 ! ! 355 ! ! 356 +--------------------------------------+ 357 358 A software rotation correction of 180 degrees should be applied to 359 correctly display the image: 360 361 +--------------------------------------+ 362 ! ! 363 ! ! 364 ! ! 365 ! |\____)\___ ! 366 ! ) _____ __`< ! 367 ! |/ )/ ! 368 ! ! 369 ! ! 370 ! ! 371 +--------------------------------------+ 372 373 Example two - Phone camera 374 375 A camera installed on the back side of a mobile device facing away from 376 the user. The captured images are meant to be displayed in portrait mode 377 (height > width) to match the device screen orientation and the device 378 usage orientation used when taking the picture. 379 380 The camera sensor is typically mounted with its pixel array longer side 381 aligned to the device longer side, upside-down mounted to compensate for 382 the lens optical inversion effect: 383 384 0 Y-Rc 385 0 +--------------------> 386 ! Y-Rp 387 ! ^ 388 ! ! 389 ! ! 390 ! ! 391 ! ! |\_____)\__ 392 ! ! ) ____ ___.< 393 ! ! |/ )/ 394 ! ! 395 ! ! 396 ! ! 397 ! 0 +-------------------------------------> 398 ! 0 X-Rp 399 ! 400 ! 401 ! 402 ! 403 V 404 X-Rc 405 406 The two reference systems are not aligned and the 'Rp' reference 407 system is rotated by 90 degrees in the counter-clockwise direction 408 relatively to the 'Rc' reference system. 409 410 The image once captured to memory will be rotated: 411 412 +-------------------------------------+ 413 | _ _ | 414 | \ / | 415 | | | | 416 | | | | 417 | | > | 418 | < | | 419 | | | | 420 | . | 421 | V | 422 +-------------------------------------+ 423 424 A correction of 90 degrees in counter-clockwise direction has to be 425 applied to correctly display the image in portrait mode on the device 426 screen: 427 428 +--------------------+ 429 | | 430 | | 431 | | 432 | | 433 | | 434 | | 435 | |\____)\___ | 436 | ) _____ __`< | 437 | |/ )/ | 438 | | 439 | | 440 | | 441 | | 442 | | 443 +--------------------+ 444 445- orientation: The orientation of a device (typically an image sensor or a flash 446 LED) describing its mounting position relative to the usage orientation of the 447 system where the device is installed on. 448 Possible values are: 449 0 - Front. The device is mounted on the front facing side of the system. 450 For mobile devices such as smartphones, tablets and laptops the front side is 451 the user facing side. 452 1 - Back. The device is mounted on the back side of the system, which is 453 defined as the opposite side of the front facing one. 454 2 - External. The device is not attached directly to the system but is 455 attached in a way that allows it to move freely. 456 457Optional endpoint properties 458---------------------------- 459 460- remote-endpoint: phandle to an 'endpoint' subnode of a remote device node. 461- slave-mode: a boolean property indicating that the link is run in slave mode. 462 The default when this property is not specified is master mode. In the slave 463 mode horizontal and vertical synchronization signals are provided to the 464 slave device (data source) by the master device (data sink). In the master 465 mode the data source device is also the source of the synchronization signals. 466- bus-type: data bus type. Possible values are: 467 1 - MIPI CSI-2 C-PHY 468 2 - MIPI CSI1 469 3 - CCP2 470 4 - MIPI CSI-2 D-PHY 471 5 - Parallel 472 6 - Bt.656 473- bus-width: number of data lines actively used, valid for the parallel busses. 474- data-shift: on the parallel data busses, if bus-width is used to specify the 475 number of data lines, data-shift can be used to specify which data lines are 476 used, e.g. "bus-width=<8>; data-shift=<2>;" means, that lines 9:2 are used. 477- hsync-active: active state of the HSYNC signal, 0/1 for LOW/HIGH respectively. 478- vsync-active: active state of the VSYNC signal, 0/1 for LOW/HIGH respectively. 479 Note, that if HSYNC and VSYNC polarities are not specified, embedded 480 synchronization may be required, where supported. 481- data-active: similar to HSYNC and VSYNC, specifies data line polarity. 482- data-enable-active: similar to HSYNC and VSYNC, specifies the data enable 483 signal polarity. 484- field-even-active: field signal level during the even field data transmission. 485- pclk-sample: sample data on rising (1) or falling (0) edge of the pixel clock 486 signal. 487- sync-on-green-active: active state of Sync-on-green (SoG) signal, 0/1 for 488 LOW/HIGH respectively. 489- data-lanes: an array of physical data lane indexes. Position of an entry 490 determines the logical lane number, while the value of an entry indicates 491 physical lane, e.g. for 2-lane MIPI CSI-2 bus we could have 492 "data-lanes = <1 2>;", assuming the clock lane is on hardware lane 0. 493 If the hardware does not support lane reordering, monotonically 494 incremented values shall be used from 0 or 1 onwards, depending on 495 whether or not there is also a clock lane. This property is valid for 496 serial busses only (e.g. MIPI CSI-2). 497- clock-lanes: an array of physical clock lane indexes. Position of an entry 498 determines the logical lane number, while the value of an entry indicates 499 physical lane, e.g. for a MIPI CSI-2 bus we could have "clock-lanes = <0>;", 500 which places the clock lane on hardware lane 0. This property is valid for 501 serial busses only (e.g. MIPI CSI-2). Note that for the MIPI CSI-2 bus this 502 array contains only one entry. 503- clock-noncontinuous: a boolean property to allow MIPI CSI-2 non-continuous 504 clock mode. 505- link-frequencies: Allowed data bus frequencies. For MIPI CSI-2, for 506 instance, this is the actual frequency of the bus, not bits per clock per 507 lane value. An array of 64-bit unsigned integers. 508- lane-polarities: an array of polarities of the lanes starting from the clock 509 lane and followed by the data lanes in the same order as in data-lanes. 510 Valid values are 0 (normal) and 1 (inverted). The length of the array 511 should be the combined length of data-lanes and clock-lanes properties. 512 If the lane-polarities property is omitted, the value must be interpreted 513 as 0 (normal). This property is valid for serial busses only. 514- strobe: Whether the clock signal is used as clock (0) or strobe (1). Used 515 with CCP2, for instance. 516 517Example 518------- 519 520The example snippet below describes two data pipelines. ov772x and imx074 are 521camera sensors with a parallel and serial (MIPI CSI-2) video bus respectively. 522Both sensors are on the I2C control bus corresponding to the i2c0 controller 523node. ov772x sensor is linked directly to the ceu0 video host interface. 524imx074 is linked to ceu0 through the MIPI CSI-2 receiver (csi2). ceu0 has a 525(single) DMA engine writing captured data to memory. ceu0 node has a single 526'port' node which may indicate that at any time only one of the following data 527pipelines can be active: ov772x -> ceu0 or imx074 -> csi2 -> ceu0. 528 529 ceu0: ceu@fe910000 { 530 compatible = "renesas,sh-mobile-ceu"; 531 reg = <0xfe910000 0xa0>; 532 interrupts = <0x880>; 533 534 mclk: master_clock { 535 compatible = "renesas,ceu-clock"; 536 #clock-cells = <1>; 537 clock-frequency = <50000000>; /* Max clock frequency */ 538 clock-output-names = "mclk"; 539 }; 540 541 port { 542 #address-cells = <1>; 543 #size-cells = <0>; 544 545 /* Parallel bus endpoint */ 546 ceu0_1: endpoint@1 { 547 reg = <1>; /* Local endpoint # */ 548 remote = <&ov772x_1_1>; /* Remote phandle */ 549 bus-width = <8>; /* Used data lines */ 550 data-shift = <2>; /* Lines 9:2 are used */ 551 552 /* If hsync-active/vsync-active are missing, 553 embedded BT.656 sync is used */ 554 hsync-active = <0>; /* Active low */ 555 vsync-active = <0>; /* Active low */ 556 data-active = <1>; /* Active high */ 557 pclk-sample = <1>; /* Rising */ 558 }; 559 560 /* MIPI CSI-2 bus endpoint */ 561 ceu0_0: endpoint@0 { 562 reg = <0>; 563 remote = <&csi2_2>; 564 }; 565 }; 566 }; 567 568 i2c0: i2c@fff20000 { 569 ... 570 ov772x_1: camera@21 { 571 compatible = "ovti,ov772x"; 572 reg = <0x21>; 573 vddio-supply = <®ulator1>; 574 vddcore-supply = <®ulator2>; 575 576 clock-frequency = <20000000>; 577 clocks = <&mclk 0>; 578 clock-names = "xclk"; 579 580 port { 581 /* With 1 endpoint per port no need for addresses. */ 582 ov772x_1_1: endpoint { 583 bus-width = <8>; 584 remote-endpoint = <&ceu0_1>; 585 hsync-active = <1>; 586 vsync-active = <0>; /* Who came up with an 587 inverter here ?... */ 588 data-active = <1>; 589 pclk-sample = <1>; 590 }; 591 }; 592 }; 593 594 imx074: camera@1a { 595 compatible = "sony,imx074"; 596 reg = <0x1a>; 597 vddio-supply = <®ulator1>; 598 vddcore-supply = <®ulator2>; 599 600 clock-frequency = <30000000>; /* Shared clock with ov772x_1 */ 601 clocks = <&mclk 0>; 602 clock-names = "sysclk"; /* Assuming this is the 603 name in the datasheet */ 604 port { 605 imx074_1: endpoint { 606 clock-lanes = <0>; 607 data-lanes = <1 2>; 608 remote-endpoint = <&csi2_1>; 609 }; 610 }; 611 }; 612 }; 613 614 csi2: csi2@ffc90000 { 615 compatible = "renesas,sh-mobile-csi2"; 616 reg = <0xffc90000 0x1000>; 617 interrupts = <0x17a0>; 618 #address-cells = <1>; 619 #size-cells = <0>; 620 621 port@1 { 622 compatible = "renesas,csi2c"; /* One of CSI2I and CSI2C. */ 623 reg = <1>; /* CSI-2 PHY #1 of 2: PHY_S, 624 PHY_M has port address 0, 625 is unused. */ 626 csi2_1: endpoint { 627 clock-lanes = <0>; 628 data-lanes = <2 1>; 629 remote-endpoint = <&imx074_1>; 630 }; 631 }; 632 port@2 { 633 reg = <2>; /* port 2: link to the CEU */ 634 635 csi2_2: endpoint { 636 remote-endpoint = <&ceu0_0>; 637 }; 638 }; 639 }; 640