1======================= 2Linux UVC Gadget Driver 3======================= 4 5Overview 6-------- 7The UVC Gadget driver is a driver for hardware on the *device* side of a USB 8connection. It is intended to run on a Linux system that has USB device-side 9hardware such as boards with an OTG port. 10 11On the device system, once the driver is bound it appears as a V4L2 device with 12the output capability. 13 14On the host side (once connected via USB cable), a device running the UVC Gadget 15driver *and controlled by an appropriate userspace program* should appear as a UVC 16specification compliant camera, and function appropriately with any program 17designed to handle them. The userspace program running on the device system can 18queue image buffers from a variety of sources to be transmitted via the USB 19connection. Typically this would mean forwarding the buffers from a camera sensor 20peripheral, but the source of the buffer is entirely dependent on the userspace 21companion program. 22 23Configuring the device kernel 24----------------------------- 25The Kconfig options USB_CONFIGFS, USB_LIBCOMPOSITE, USB_CONFIGFS_F_UVC and 26USB_F_UVC must be selected to enable support for the UVC gadget. 27 28Configuring the gadget through configfs 29--------------------------------------- 30The UVC Gadget expects to be configured through configfs using the UVC function. 31This allows a significant degree of flexibility, as many of a UVC device's 32settings can be controlled this way. 33 34Not all of the available attributes are described here. For a complete enumeration 35see Documentation/ABI/testing/configfs-usb-gadget-uvc 36 37Assumptions 38~~~~~~~~~~~ 39This section assumes that you have mounted configfs at `/sys/kernel/config` and 40created a gadget as `/sys/kernel/config/usb_gadget/g1`. 41 42The UVC Function 43~~~~~~~~~~~~~~~~ 44 45The first step is to create the UVC function: 46 47.. code-block:: bash 48 49 # These variables will be assumed throughout the rest of the document 50 CONFIGFS="/sys/kernel/config" 51 GADGET="$CONFIGFS/usb_gadget/g1" 52 FUNCTION="$GADGET/functions/uvc.0" 53 54 mkdir -p $FUNCTION 55 56Formats and Frames 57~~~~~~~~~~~~~~~~~~ 58 59You must configure the gadget by telling it which formats you support, as well 60as the frame sizes and frame intervals that are supported for each format. In 61the current implementation there is no way for the gadget to refuse to set a 62format that the host instructs it to set, so it is important that this step is 63completed *accurately* to ensure that the host never asks for a format that 64can't be provided. 65 66Formats are created under the streaming/uncompressed and streaming/mjpeg configfs 67groups, with the framesizes created under the formats in the following 68structure: 69 70:: 71 72 uvc.0 + 73 | 74 + streaming + 75 | 76 + mjpeg + 77 | | 78 | + mjpeg + 79 | | 80 | + 720p 81 | | 82 | + 1080p 83 | 84 + uncompressed + 85 | 86 + yuyv + 87 | 88 + 720p 89 | 90 + 1080p 91 92Each frame can then be configured with a width and height, plus the maximum 93buffer size required to store a single frame, and finally with the supported 94frame intervals for that format and framesize. Width and height are enumerated in 95units of pixels, frame interval in units of 100ns. To create the structure 96above with 2, 15 and 100 fps frameintervals for each framesize for example you 97might do: 98 99.. code-block:: bash 100 101 create_frame() { 102 # Example usage: 103 # create_frame <width> <height> <group> <format name> 104 105 WIDTH=$1 106 HEIGHT=$2 107 FORMAT=$3 108 NAME=$4 109 110 wdir=$FUNCTION/streaming/$FORMAT/$NAME/${HEIGHT}p 111 112 mkdir -p $wdir 113 echo $WIDTH > $wdir/wWidth 114 echo $HEIGHT > $wdir/wHeight 115 echo $(( $WIDTH * $HEIGHT * 2 )) > $wdir/dwMaxVideoFrameBufferSize 116 cat <<EOF > $wdir/dwFrameInterval 117 666666 118 100000 119 5000000 120 EOF 121 } 122 123 create_frame 1280 720 mjpeg mjpeg 124 create_frame 1920 1080 mjpeg mjpeg 125 create_frame 1280 720 uncompressed yuyv 126 create_frame 1920 1080 uncompressed yuyv 127 128The only uncompressed format currently supported is YUYV, which is detailed at 129Documentation/userspace-api/media/v4l/pixfmt-packed.yuv.rst. 130 131Color Matching Descriptors 132~~~~~~~~~~~~~~~~~~~~~~~~~~ 133It's possible to specify some colometry information for each format you create. 134This step is optional, and default information will be included if this step is 135skipped; those default values follow those defined in the Color Matching Descriptor 136section of the UVC specification. 137 138To create a Color Matching Descriptor, create a configfs item and set its three 139attributes to your desired settings and then link to it from the format you wish 140it to be associated with: 141 142.. code-block:: bash 143 144 # Create a new Color Matching Descriptor 145 146 mkdir $FUNCTION/streaming/color_matching/yuyv 147 pushd $FUNCTION/streaming/color_matching/yuyv 148 149 echo 1 > bColorPrimaries 150 echo 1 > bTransferCharacteristics 151 echo 4 > bMatrixCoefficients 152 153 popd 154 155 # Create a symlink to the Color Matching Descriptor from the format's config item 156 ln -s $FUNCTION/streaming/color_matching/yuyv $FUNCTION/streaming/uncompressed/yuyv 157 158For details about the valid values, consult the UVC specification. Note that a 159default color matching descriptor exists and is used by any format which does 160not have a link to a different Color Matching Descriptor. It's possible to 161change the attribute settings for the default descriptor, so bear in mind that if 162you do that you are altering the defaults for any format that does not link to 163a different one. 164 165 166Header linking 167~~~~~~~~~~~~~~ 168 169The UVC specification requires that Format and Frame descriptors be preceded by 170Headers detailing things such as the number and cumulative size of the different 171Format descriptors that follow. This and similar operations are acheived in 172configfs by linking between the configfs item representing the header and the 173config items representing those other descriptors, in this manner: 174 175.. code-block:: bash 176 177 mkdir $FUNCTION/streaming/header/h 178 179 # This section links the format descriptors and their associated frames 180 # to the header 181 cd $FUNCTION/streaming/header/h 182 ln -s ../../uncompressed/yuyv 183 ln -s ../../mjpeg/mjpeg 184 185 # This section ensures that the header will be transmitted for each 186 # speed's set of descriptors. If support for a particular speed is not 187 # needed then it can be skipped here. 188 cd ../../class/fs 189 ln -s ../../header/h 190 cd ../../class/hs 191 ln -s ../../header/h 192 cd ../../class/ss 193 ln -s ../../header/h 194 cd ../../../control 195 mkdir header/h 196 ln -s header/h class/fs 197 ln -s header/h class/ss 198 199 200Extension Unit Support 201~~~~~~~~~~~~~~~~~~~~~~ 202 203A UVC Extension Unit (XU) basically provides a distinct unit to which control set 204and get requests can be addressed. The meaning of those control requests is 205entirely implementation dependent, but may be used to control settings outside 206of the UVC specification (for example enabling or disabling video effects). An 207XU can be inserted into the UVC unit chain or left free-hanging. 208 209Configuring an extension unit involves creating an entry in the appropriate 210directory and setting its attributes appropriately, like so: 211 212.. code-block:: bash 213 214 mkdir $FUNCTION/control/extensions/xu.0 215 pushd $FUNCTION/control/extensions/xu.0 216 217 # Set the bUnitID of the Processing Unit as the source for this 218 # Extension Unit 219 echo 2 > baSourceID 220 221 # Set this XU as the source of the default output terminal. This inserts 222 # the XU into the UVC chain between the PU and OT such that the final 223 # chain is IT > PU > XU.0 > OT 224 cat bUnitID > ../../terminal/output/default/baSourceID 225 226 # Flag some controls as being available for use. The bmControl field is 227 # a bitmap with each bit denoting the availability of a particular 228 # control. For example to flag the 0th, 2nd and 3rd controls available: 229 echo 0x0d > bmControls 230 231 # Set the GUID; this is a vendor-specific code identifying the XU. 232 echo -e -n "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10" > guidExtensionCode 233 234 popd 235 236The bmControls attribute and the baSourceID attribute are multi-value attributes. 237This means that you may write multiple newline separated values to them. For 238example to flag the 1st, 2nd, 9th and 10th controls as being available you would 239need to write two values to bmControls, like so: 240 241.. code-block:: bash 242 243 cat << EOF > bmControls 244 0x03 245 0x03 246 EOF 247 248The multi-value nature of the baSourceID attribute belies the fact that XUs can 249be multiple-input, though note that this currently has no significant effect. 250 251The bControlSize attribute reflects the size of the bmControls attribute, and 252similarly bNrInPins reflects the size of the baSourceID attributes. Both 253attributes are automatically increased / decreased as you set bmControls and 254baSourceID. It is also possible to manually increase or decrease bControlSize 255which has the effect of truncating entries to the new size, or padding entries 256out with 0x00, for example: 257 258:: 259 260 $ cat bmControls 261 0x03 262 0x05 263 264 $ cat bControlSize 265 2 266 267 $ echo 1 > bControlSize 268 $ cat bmControls 269 0x03 270 271 $ echo 2 > bControlSize 272 $ cat bmControls 273 0x03 274 0x00 275 276bNrInPins and baSourceID function in the same way. 277 278Custom Strings Support 279~~~~~~~~~~~~~~~~~~~~~~ 280 281String descriptors that provide a textual description for various parts of a 282USB device can be defined in the usual place within USB configfs, and may then 283be linked to from the UVC function root or from Extension Unit directories to 284assign those strings as descriptors: 285 286.. code-block:: bash 287 288 # Create a string descriptor in us-EN and link to it from the function 289 # root. The name of the link is significant here, as it declares this 290 # descriptor to be intended for the Interface Association Descriptor. 291 # Other significant link names at function root are vs0_desc and vs1_desc 292 # For the VideoStreaming Interface 0/1 Descriptors. 293 294 mkdir -p $GADGET/strings/0x409/iad_desc 295 echo -n "Interface Associaton Descriptor" > $GADGET/strings/0x409/iad_desc/s 296 ln -s $GADGET/strings/0x409/iad_desc $FUNCTION/iad_desc 297 298 # Because the link to a String Descriptor from an Extension Unit clearly 299 # associates the two, the name of this link is not significant and may 300 # be set freely. 301 302 mkdir -p $GADGET/strings/0x409/xu.0 303 echo -n "A Very Useful Extension Unit" > $GADGET/strings/0x409/xu.0/s 304 ln -s $GADGET/strings/0x409/xu.0 $FUNCTION/control/extensions/xu.0 305 306The interrupt endpoint 307~~~~~~~~~~~~~~~~~~~~~~ 308 309The VideoControl interface has an optional interrupt endpoint which is by default 310disabled. This is intended to support delayed response control set requests for 311UVC (which should respond through the interrupt endpoint rather than tying up 312endpoint 0). At present support for sending data through this endpoint is missing 313and so it is left disabled to avoid confusion. If you wish to enable it you can 314do so through the configfs attribute: 315 316.. code-block:: bash 317 318 echo 1 > $FUNCTION/control/enable_interrupt_ep 319 320Bandwidth configuration 321~~~~~~~~~~~~~~~~~~~~~~~ 322 323There are three attributes which control the bandwidth of the USB connection. 324These live in the function root and can be set within limits: 325 326.. code-block:: bash 327 328 # streaming_interval sets bInterval. Values range from 1..255 329 echo 1 > $FUNCTION/streaming_interval 330 331 # streaming_maxpacket sets wMaxPacketSize. Valid values are 1024/2048/3072 332 echo 3072 > $FUNCTION/streaming_maxpacket 333 334 # streaming_maxburst sets bMaxBurst. Valid values are 1..15 335 echo 1 > $FUNCTION/streaming_maxburst 336 337 338The values passed here will be clamped to valid values according to the UVC 339specification (which depend on the speed of the USB connection). To understand 340how the settings influence bandwidth you should consult the UVC specifications, 341but a rule of thumb is that increasing the streaming_maxpacket setting will 342improve bandwidth (and thus the maximum possible framerate), whilst the same is 343true for streaming_maxburst provided the USB connection is running at SuperSpeed. 344Increasing streaming_interval will reduce bandwidth and framerate. 345 346The userspace application 347------------------------- 348By itself, the UVC Gadget driver cannot do anything particularly interesting. It 349must be paired with a userspace program that responds to UVC control requests and 350fills buffers to be queued to the V4L2 device that the driver creates. How those 351things are achieved is implementation dependent and beyond the scope of this 352document, but a reference application can be found at https://gitlab.freedesktop.org/camera/uvc-gadget 353