1.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
2
3.. _v4l2-meta-fmt-d4xx:
4
5*******************************
6V4L2_META_FMT_D4XX ('D4XX')
7*******************************
8
9Intel D4xx UVC Cameras Metadata
10
11
12Description
13===========
14
15Intel D4xx (D435, D455 and others) cameras include per-frame metadata in their UVC
16payload headers, following the Microsoft(R) UVC extension proposal [1_]. That
17means, that the private D4XX metadata, following the standard UVC header, is
18organised in blocks. D4XX cameras implement several standard block types,
19proposed by Microsoft, and several proprietary ones. Supported standard metadata
20types are MetadataId_CaptureStats (ID 3), MetadataId_CameraExtrinsics (ID 4),
21and MetadataId_CameraIntrinsics (ID 5). For their description see [1_]. This
22document describes proprietary metadata types, used by D4xx cameras.
23
24V4L2_META_FMT_D4XX buffers follow the metadata buffer layout of
25V4L2_META_FMT_UVC with the only difference, that it also includes proprietary
26payload header data. D4xx cameras use bulk transfers and only send one payload
27per frame, therefore their headers cannot be larger than 255 bytes.
28
29This document implements Intel Configuration version 3 [9_].
30
31Below are proprietary Microsoft style metadata types, used by D4xx cameras,
32where all fields are in little endian order:
33
34.. tabularcolumns:: |p{5.0cm}|p{12.5cm}|
35
36
37.. flat-table:: D4xx metadata
38    :widths: 1 2
39    :header-rows:  1
40    :stub-columns: 0
41
42    * - **Field**
43      - **Description**
44    * - :cspan:`1` *Depth Control*
45    * - __u32 ID
46      - 0x80000000
47    * - __u32 Size
48      - Size in bytes, include ID (all protocol versions: 60)
49    * - __u32 Version
50      - Version of this structure. The documentation herein covers versions 1,
51        2 and 3. The version number will be incremented when new fields are
52        added.
53    * - __u32 Flags
54      - A bitmask of flags: see [2_] below
55    * - __u32 Gain
56      - Gain value in internal units, same as the V4L2_CID_GAIN control, used to
57	capture the frame
58    * - __u32 Exposure
59      - Exposure time (in microseconds) used to capture the frame
60    * - __u32 Laser power
61      - Power of the laser LED 0-360, used for depth measurement
62    * - __u32 AE mode
63      - 0: manual; 1: automatic exposure
64    * - __u32 Exposure priority
65      - Exposure priority value: 0 - constant frame rate
66    * - __u32 AE ROI left
67      - Left border of the AE Region of Interest (all ROI values are in pixels
68	and lie between 0 and maximum width or height respectively)
69    * - __u32 AE ROI right
70      - Right border of the AE Region of Interest
71    * - __u32 AE ROI top
72      - Top border of the AE Region of Interest
73    * - __u32 AE ROI bottom
74      - Bottom border of the AE Region of Interest
75    * - __u32 Preset
76      - Preset selector value, default: 0, unless changed by the user
77    * - __u8 Emitter mode (v3 only) (__u32 Laser mode for v1) [8_]
78      - 0: off, 1: on, same as __u32 Laser mode for v1
79    * - __u8 RFU byte (v3 only)
80      - Spare byte for future use
81    * - __u16 LED Power (v3 only)
82      - Led power value 0-360 (F416 SKU)
83    * - :cspan:`1` *Capture Timing*
84    * - __u32 ID
85      - 0x80000001
86    * - __u32 Size
87      - Size in bytes, include ID (all protocol versions: 40)
88    * - __u32 Version
89      - Version of this structure. The documentation herein corresponds to
90        version xxx. The version number will be incremented when new fields are
91        added.
92    * - __u32 Flags
93      - A bitmask of flags: see [3_] below
94    * - __u32 Frame counter
95      - Monotonically increasing counter
96    * - __u32 Optical time
97      - Time in microseconds from the beginning of a frame till its middle
98    * - __u32 Readout time
99      - Time, used to read out a frame in microseconds
100    * - __u32 Exposure time
101      - Frame exposure time in microseconds
102    * - __u32 Frame interval
103      - In microseconds = 1000000 / framerate
104    * - __u32 Pipe latency
105      - Time in microseconds from start of frame to data in USB buffer
106    * - :cspan:`1` *Configuration*
107    * - __u32 ID
108      - 0x80000002
109    * - __u32 Size
110      - Size in bytes, include ID (v1:36, v3:40)
111    * - __u32 Version
112      - Version of this structure. The documentation herein corresponds to
113        version xxx. The version number will be incremented when new fields are
114        added.
115    * - __u32 Flags
116      - A bitmask of flags: see [4_] below
117    * - __u8 Hardware type
118      - Camera hardware version [5_]
119    * - __u8 SKU ID
120      - Camera hardware configuration [6_]
121    * - __u32 Cookie
122      - Internal synchronisation
123    * - __u16 Format
124      - Image format code [7_]
125    * - __u16 Width
126      - Width in pixels
127    * - __u16 Height
128      - Height in pixels
129    * - __u16 Framerate
130      - Requested frame rate per second
131    * - __u16 Trigger
132      - Byte 0: bit 0: depth and RGB are synchronised, bit 1: external trigger
133    * - __u16 Calibration count (v3 only)
134      - Calibration counter, see [4_] below
135    * - __u8 GPIO input data (v3 only)
136      - GPIO readout, see [4_] below (Supported from FW 5.12.7.0)
137    * - __u32 Sub-preset info (v3 only)
138      - Sub-preset choice information, see [4_] below
139    * - __u8 reserved (v3 only)
140      - RFU byte.
141
142.. _1:
143
144[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/uvc-extensions-1-5
145
146.. _2:
147
148[2] Depth Control flags specify which fields are valid: ::
149
150  0x00000001 Gain
151  0x00000002 Exposure
152  0x00000004 Laser power
153  0x00000008 AE mode
154  0x00000010 Exposure priority
155  0x00000020 AE ROI
156  0x00000040 Preset
157  0x00000080 Emitter mode
158  0x00000100 LED Power
159
160.. _3:
161
162[3] Capture Timing flags specify which fields are valid: ::
163
164  0x00000001 Frame counter
165  0x00000002 Optical time
166  0x00000004 Readout time
167  0x00000008 Exposure time
168  0x00000010 Frame interval
169  0x00000020 Pipe latency
170
171.. _4:
172
173[4] Configuration flags specify which fields are valid: ::
174
175  0x00000001 Hardware type
176  0x00000002 SKU ID
177  0x00000004 Cookie
178  0x00000008 Format
179  0x00000010 Width
180  0x00000020 Height
181  0x00000040 Framerate
182  0x00000080 Trigger
183  0x00000100 Cal count
184  0x00000200 GPIO Input Data
185  0x00000400 Sub-preset Info
186
187.. _5:
188
189[5] Camera model: ::
190
191  0 DS5
192  1 IVCAM2
193
194.. _6:
195
196[6] 8-bit camera hardware configuration bitfield: ::
197
198  [1:0] depthCamera
199	00: no depth
200	01: standard depth
201	10: wide depth
202	11: reserved
203  [2]   depthIsActive - has a laser projector
204  [3]   RGB presence
205  [4]   Inertial Measurement Unit (IMU) presence
206  [5]   projectorType
207	0: HPTG
208	1: Princeton
209  [6]   0: a projector, 1: an LED
210  [7]   reserved
211
212.. _7:
213
214[7] Image format codes per video streaming interface:
215
216Depth: ::
217
218  1 Z16
219  2 Z
220
221Left sensor: ::
222
223  1 Y8
224  2 UYVY
225  3 R8L8
226  4 Calibration
227  5 W10
228
229Fish Eye sensor: ::
230
231  1 RAW8
232
233.. _8:
234
235[8] The "Laser mode" has been replaced in version 3 by three different fields.
236"Laser" has been renamed to "Emitter" as there are multiple technologies for
237camera projectors. As we have another field for "Laser Power" we introduced
238"LED Power" for extra emitter.
239
240The "Laser mode" __u32 fiels has been split into: ::
241   1 __u8 Emitter mode
242   2 __u8 RFU byte
243   3 __u16 LED Power
244
245This is a change between versions 1 and 3. All versions 1, 2 and 3 are backward
246compatible with the same data format and they are supported. See [2_] for which
247attributes are valid.
248
249.. _9:
250
251[9] LibRealSense SDK metadata source:
252https://github.com/IntelRealSense/librealsense/blob/master/src/metadata.h
253