1=======================
2Linux UVC Gadget Driver
3=======================
4
5Overview
6--------
7The UVC Gadget driver is a driver for hardware on the *device* side of a USB
8connection. It is intended to run on a Linux system that has USB device-side
9hardware such as boards with an OTG port.
10
11On the device system, once the driver is bound it appears as a V4L2 device with
12the output capability.
13
14On the host side (once connected via USB cable), a device running the UVC Gadget
15driver *and controlled by an appropriate userspace program* should appear as a UVC
16specification compliant camera, and function appropriately with any program
17designed to handle them. The userspace program running on the device system can
18queue image buffers from a variety of sources to be transmitted via the USB
19connection. Typically this would mean forwarding the buffers from a camera sensor
20peripheral, but the source of the buffer is entirely dependent on the userspace
21companion program.
22
23Configuring the device kernel
24-----------------------------
25The Kconfig options USB_CONFIGFS, USB_LIBCOMPOSITE, USB_CONFIGFS_F_UVC and
26USB_F_UVC must be selected to enable support for the UVC gadget.
27
28Configuring the gadget through configfs
29---------------------------------------
30The UVC Gadget expects to be configured through configfs using the UVC function.
31This allows a significant degree of flexibility, as many of a UVC device's
32settings can be controlled this way.
33
34Not all of the available attributes are described here. For a complete enumeration
35see Documentation/ABI/testing/configfs-usb-gadget-uvc
36
37Assumptions
38~~~~~~~~~~~
39This section assumes that you have mounted configfs at `/sys/kernel/config` and
40created a gadget as `/sys/kernel/config/usb_gadget/g1`.
41
42The UVC Function
43~~~~~~~~~~~~~~~~
44
45The first step is to create the UVC function:
46
47.. code-block:: bash
48
49	# These variables will be assumed throughout the rest of the document
50	CONFIGFS="/sys/kernel/config"
51	GADGET="$CONFIGFS/usb_gadget/g1"
52	FUNCTION="$GADGET/functions/uvc.0"
53
54	mkdir -p $FUNCTION
55
56Formats and Frames
57~~~~~~~~~~~~~~~~~~
58
59You must configure the gadget by telling it which formats you support, as well
60as the frame sizes and frame intervals that are supported for each format. In
61the current implementation there is no way for the gadget to refuse to set a
62format that the host instructs it to set, so it is important that this step is
63completed *accurately* to ensure that the host never asks for a format that
64can't be provided.
65
66Formats are created under the streaming/uncompressed and streaming/mjpeg configfs
67groups, with the framesizes created under the formats in the following
68structure:
69
70::
71
72	uvc.0 +
73	      |
74	      + streaming +
75			  |
76			  + mjpeg +
77			  |       |
78			  |       + mjpeg +
79			  |	       |
80			  |	       + 720p
81			  |	       |
82			  |	       + 1080p
83			  |
84			  + uncompressed +
85					 |
86					 + yuyv +
87						|
88						+ 720p
89						|
90						+ 1080p
91
92Each frame can then be configured with a width and height, plus the maximum
93buffer size required to store a single frame, and finally with the supported
94frame intervals for that format and framesize. Width and height are enumerated in
95units of pixels, frame interval in units of 100ns. To create the structure
96above with 2, 15 and 100 fps frameintervals for each framesize for example you
97might do:
98
99.. code-block:: bash
100
101	create_frame() {
102		# Example usage:
103		# create_frame <width> <height> <group> <format name>
104
105		WIDTH=$1
106		HEIGHT=$2
107		FORMAT=$3
108		NAME=$4
109
110		wdir=$FUNCTION/streaming/$FORMAT/$NAME/${HEIGHT}p
111
112		mkdir -p $wdir
113		echo $WIDTH > $wdir/wWidth
114		echo $HEIGHT > $wdir/wHeight
115		echo $(( $WIDTH * $HEIGHT * 2 )) > $wdir/dwMaxVideoFrameBufferSize
116		cat <<EOF > $wdir/dwFrameInterval
117	666666
118	100000
119	5000000
120	EOF
121	}
122
123	create_frame 1280 720 mjpeg mjpeg
124	create_frame 1920 1080 mjpeg mjpeg
125	create_frame 1280 720 uncompressed yuyv
126	create_frame 1920 1080 uncompressed yuyv
127
128The only uncompressed format currently supported is YUYV, which is detailed at
129Documentation/userspace-api/media/v4l/pixfmt-packed.yuv.rst.
130
131Color Matching Descriptors
132~~~~~~~~~~~~~~~~~~~~~~~~~~
133It's possible to specify some colometry information for each format you create.
134This step is optional, and default information will be included if this step is
135skipped; those default values follow those defined in the Color Matching Descriptor
136section of the UVC specification.
137
138To create a Color Matching Descriptor, create a configfs item and set its three
139attributes to your desired settings and then link to it from the format you wish
140it to be associated with:
141
142.. code-block:: bash
143
144	# Create a new Color Matching Descriptor
145
146	mkdir $FUNCTION/streaming/color_matching/yuyv
147	pushd $FUNCTION/streaming/color_matching/yuyv
148
149	echo 1 > bColorPrimaries
150	echo 1 > bTransferCharacteristics
151	echo 4 > bMatrixCoefficients
152
153	popd
154
155	# Create a symlink to the Color Matching Descriptor from the format's config item
156	ln -s $FUNCTION/streaming/color_matching/yuyv $FUNCTION/streaming/uncompressed/yuyv
157
158For details about the valid values, consult the UVC specification. Note that a
159default color matching descriptor exists and is used by any format which does
160not have a link to a different Color Matching Descriptor. It's possible to
161change the attribute settings for the default descriptor, so bear in mind that if
162you do that you are altering the defaults for any format that does not link to
163a different one.
164
165
166Header linking
167~~~~~~~~~~~~~~
168
169The UVC specification requires that Format and Frame descriptors be preceded by
170Headers detailing things such as the number and cumulative size of the different
171Format descriptors that follow. This and similar operations are achieved in
172configfs by linking between the configfs item representing the header and the
173config items representing those other descriptors, in this manner:
174
175.. code-block:: bash
176
177	mkdir $FUNCTION/streaming/header/h
178
179	# This section links the format descriptors and their associated frames
180	# to the header
181	cd $FUNCTION/streaming/header/h
182	ln -s ../../uncompressed/yuyv
183	ln -s ../../mjpeg/mjpeg
184
185	# This section ensures that the header will be transmitted for each
186	# speed's set of descriptors. If support for a particular speed is not
187	# needed then it can be skipped here.
188	cd ../../class/fs
189	ln -s ../../header/h
190	cd ../../class/hs
191	ln -s ../../header/h
192	cd ../../class/ss
193	ln -s ../../header/h
194	cd ../../../control
195	mkdir header/h
196	ln -s header/h class/fs
197	ln -s header/h class/ss
198
199
200Extension Unit Support
201~~~~~~~~~~~~~~~~~~~~~~
202
203A UVC Extension Unit (XU) basically provides a distinct unit to which control set
204and get requests can be addressed. The meaning of those control requests is
205entirely implementation dependent, but may be used to control settings outside
206of the UVC specification (for example enabling or disabling video effects). An
207XU can be inserted into the UVC unit chain or left free-hanging.
208
209Configuring an extension unit involves creating an entry in the appropriate
210directory and setting its attributes appropriately, like so:
211
212.. code-block:: bash
213
214	mkdir $FUNCTION/control/extensions/xu.0
215	pushd $FUNCTION/control/extensions/xu.0
216
217	# Set the bUnitID of the Processing Unit as the source for this
218	# Extension Unit
219	echo 2 > baSourceID
220
221	# Set this XU as the source of the default output terminal. This inserts
222	# the XU into the UVC chain between the PU and OT such that the final
223	# chain is IT > PU > XU.0 > OT
224	cat bUnitID > ../../terminal/output/default/baSourceID
225
226	# Flag some controls as being available for use. The bmControl field is
227	# a bitmap with each bit denoting the availability of a particular
228	# control. For example to flag the 0th, 2nd and 3rd controls available:
229	echo 0x0d > bmControls
230
231	# Set the GUID; this is a vendor-specific code identifying the XU.
232	echo -e -n "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10" > guidExtensionCode
233
234	popd
235
236The bmControls attribute and the baSourceID attribute are multi-value attributes.
237This means that you may write multiple newline separated values to them. For
238example to flag the 1st, 2nd, 9th and 10th controls as being available you would
239need to write two values to bmControls, like so:
240
241.. code-block:: bash
242
243	cat << EOF > bmControls
244	0x03
245	0x03
246	EOF
247
248The multi-value nature of the baSourceID attribute belies the fact that XUs can
249be multiple-input, though note that this currently has no significant effect.
250
251The bControlSize attribute reflects the size of the bmControls attribute, and
252similarly bNrInPins reflects the size of the baSourceID attributes. Both
253attributes are automatically increased / decreased as you set bmControls and
254baSourceID. It is also possible to manually increase or decrease bControlSize
255which has the effect of truncating entries to the new size, or padding entries
256out with 0x00, for example:
257
258::
259
260	$ cat bmControls
261	0x03
262	0x05
263
264	$ cat bControlSize
265	2
266
267	$ echo 1 > bControlSize
268	$ cat bmControls
269	0x03
270
271	$ echo 2 > bControlSize
272	$ cat bmControls
273	0x03
274	0x00
275
276bNrInPins and baSourceID function in the same way.
277
278Configuring Supported Controls for Camera Terminal and Processing Unit
279~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
280
281The Camera Terminal and Processing Units in the UVC chain also have bmControls
282attributes which function similarly to the same field in an Extension Unit.
283Unlike XUs however, the meaning of the bitflag for these units is defined in
284the UVC specification; you should consult the "Camera Terminal Descriptor" and
285"Processing Unit Descriptor" sections for an enumeration of the flags.
286
287.. code-block:: bash
288
289        # Set the Processing Unit's bmControls, flagging Brightness, Contrast
290        # and Hue as available controls:
291        echo 0x05 > $FUNCTION/control/processing/default/bmControls
292
293        # Set the Camera Terminal's bmControls, flagging Focus Absolute and
294        # Focus Relative as available controls:
295        echo 0x60 > $FUNCTION/control/terminal/camera/default/bmControls
296
297If you do not set these fields then by default the Auto-Exposure Mode control
298for the Camera Terminal and the Brightness control for the Processing Unit will
299be flagged as available; if they are not supported you should set the field to
3000x00.
301
302Note that the size of the bmControls field for a Camera Terminal or Processing
303Unit is fixed by the UVC specification, and so the bControlSize attribute is
304read-only here.
305
306Custom Strings Support
307~~~~~~~~~~~~~~~~~~~~~~
308
309String descriptors that provide a textual description for various parts of a
310USB device can be defined in the usual place within USB configfs, and may then
311be linked to from the UVC function root or from Extension Unit directories to
312assign those strings as descriptors:
313
314.. code-block:: bash
315
316	# Create a string descriptor in us-EN and link to it from the function
317	# root. The name of the link is significant here, as it declares this
318	# descriptor to be intended for the Interface Association Descriptor.
319	# Other significant link names at function root are vs0_desc and vs1_desc
320	# For the VideoStreaming Interface 0/1 Descriptors.
321
322	mkdir -p $GADGET/strings/0x409/iad_desc
323	echo -n "Interface Associaton Descriptor" > $GADGET/strings/0x409/iad_desc/s
324	ln -s $GADGET/strings/0x409/iad_desc $FUNCTION/iad_desc
325
326	# Because the link to a String Descriptor from an Extension Unit clearly
327	# associates the two, the name of this link is not significant and may
328	# be set freely.
329
330	mkdir -p $GADGET/strings/0x409/xu.0
331	echo -n "A Very Useful Extension Unit" > $GADGET/strings/0x409/xu.0/s
332	ln -s $GADGET/strings/0x409/xu.0 $FUNCTION/control/extensions/xu.0
333
334The interrupt endpoint
335~~~~~~~~~~~~~~~~~~~~~~
336
337The VideoControl interface has an optional interrupt endpoint which is by default
338disabled. This is intended to support delayed response control set requests for
339UVC (which should respond through the interrupt endpoint rather than tying up
340endpoint 0). At present support for sending data through this endpoint is missing
341and so it is left disabled to avoid confusion. If you wish to enable it you can
342do so through the configfs attribute:
343
344.. code-block:: bash
345
346	echo 1 > $FUNCTION/control/enable_interrupt_ep
347
348Bandwidth configuration
349~~~~~~~~~~~~~~~~~~~~~~~
350
351There are three attributes which control the bandwidth of the USB connection.
352These live in the function root and can be set within limits:
353
354.. code-block:: bash
355
356	# streaming_interval sets bInterval. Values range from 1..255
357	echo 1 > $FUNCTION/streaming_interval
358
359	# streaming_maxpacket sets wMaxPacketSize. Valid values are 1024/2048/3072
360	echo 3072 > $FUNCTION/streaming_maxpacket
361
362	# streaming_maxburst sets bMaxBurst. Valid values are 1..15
363	echo 1 > $FUNCTION/streaming_maxburst
364
365
366The values passed here will be clamped to valid values according to the UVC
367specification (which depend on the speed of the USB connection). To understand
368how the settings influence bandwidth you should consult the UVC specifications,
369but a rule of thumb is that increasing the streaming_maxpacket setting will
370improve bandwidth (and thus the maximum possible framerate), whilst the same is
371true for streaming_maxburst provided the USB connection is running at SuperSpeed.
372Increasing streaming_interval will reduce bandwidth and framerate.
373
374The userspace application
375-------------------------
376By itself, the UVC Gadget driver cannot do anything particularly interesting. It
377must be paired with a userspace program that responds to UVC control requests and
378fills buffers to be queued to the V4L2 device that the driver creates. How those
379things are achieved is implementation dependent and beyond the scope of this
380document, but a reference application can be found at https://gitlab.freedesktop.org/camera/uvc-gadget
381