1PCI SR/IOV EMULATION SUPPORT 2============================ 3 4Description 5=========== 6SR/IOV (Single Root I/O Virtualization) is an optional extended capability 7of a PCI Express device. It allows a single physical function (PF) to appear as multiple 8virtual functions (VFs) for the main purpose of eliminating software 9overhead in I/O from virtual machines. 10 11QEMU now implements the basic common functionality to enable an emulated device 12to support SR/IOV. 13 14Implementation 15============== 16Implementing emulation of an SR/IOV capable device typically consists of 17implementing support for two types of device classes; the "normal" physical device 18(PF) and the virtual device (VF). From QEMU's perspective, the VFs are just 19like other devices, except that some of their properties are derived from 20the PF. 21 22A virtual function is different from a physical function in that the BAR 23space for all VFs are defined by the BAR registers in the PFs SR/IOV 24capability. All VFs have the same BARs and BAR sizes. 25 26Accesses to these virtual BARs then is computed as 27 28 <VF BAR start> + <VF number> * <BAR sz> + <offset> 29 30From our emulation perspective this means that there is a separate call for 31setting up a BAR for a VF. 32 331) To enable SR/IOV support in the PF, it must be a PCI Express device so 34 you would need to add a PCI Express capability in the normal PCI 35 capability list. You might also want to add an ARI (Alternative 36 Routing-ID Interpretation) capability to indicate that your device 37 supports functions beyond it's "own" function space (0-7), 38 which is necessary to support more than 7 functions, or 39 if functions extends beyond offset 7 because they are placed at an 40 offset > 1 or have stride > 1. 41 42 ... 43 #include "hw/pci/pcie.h" 44 #include "hw/pci/pcie_sriov.h" 45 46 pci_your_pf_dev_realize( ... ) 47 { 48 ... 49 int ret = pcie_endpoint_cap_init(d, 0x70); 50 ... 51 pcie_ari_init(d, 0x100); 52 ... 53 54 /* Add and initialize the SR/IOV capability */ 55 if (!pcie_sriov_pf_init(d, 0x200, "your_virtual_dev", 56 vf_devid, initial_vfs, total_vfs, 57 fun_offset, stride, errp)) { 58 return; 59 } 60 61 /* Set up individual VF BARs (parameters as for normal BARs) */ 62 pcie_sriov_pf_init_vf_bar( ... ) 63 ... 64 } 65 66 For cleanup, you simply call: 67 68 pcie_sriov_pf_exit(device); 69 70 which will delete all the virtual functions and associated resources. 71 722) Similarly in the implementation of the virtual function, you need to 73 make it a PCI Express device and add a similar set of capabilities 74 except for the SR/IOV capability. Then you need to set up the VF BARs as 75 subregions of the PFs SR/IOV VF BARs by calling 76 pcie_sriov_vf_register_bar() instead of the normal pci_register_bar() call: 77 78 pci_your_vf_dev_realize( ... ) 79 { 80 ... 81 int ret = pcie_endpoint_cap_init(d, 0x60); 82 ... 83 pcie_ari_init(d, 0x100); 84 ... 85 memory_region_init(mr, ... ) 86 pcie_sriov_vf_register_bar(d, bar_nr, mr); 87 ... 88 } 89 90Testing on Linux guest 91====================== 92The easiest is if your device driver supports sysfs based SR/IOV 93enabling. Support for this was added in kernel v.3.8, so not all drivers 94support it yet. 95 96To enable 4 VFs for a device at 01:00.0: 97 98 modprobe yourdriver 99 echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs 100 101You should now see 4 VFs with lspci. 102To turn SR/IOV off again - the standard requires you to turn it off before you can enable 103another VF count, and the emulation enforces this: 104 105 echo 0 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs 106 107Older drivers typically provide a max_vfs module parameter 108to enable it at load time: 109 110 modprobe yourdriver max_vfs=4 111 112To disable the VFs again then, you simply have to unload the driver: 113 114 rmmod yourdriver 115