Commit | Line | Data |
---|---|---|
15b49bee YZ |
1 | PCI Express I/O Virtualization Howto |
2 | Copyright (C) 2009 Intel Corporation | |
3 | Yu Zhao <yu.zhao@intel.com> | |
4 | ||
2597ba76 DD |
5 | Update: November 2012 |
6 | -- sysfs-based SRIOV enable-/disable-ment | |
7 | Donald Dutile <ddutile@redhat.com> | |
15b49bee YZ |
8 | |
9 | 1. Overview | |
10 | ||
11 | 1.1 What is SR-IOV | |
12 | ||
13 | Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended | |
14 | capability which makes one physical device appear as multiple virtual | |
15 | devices. The physical device is referred to as Physical Function (PF) | |
16 | while the virtual devices are referred to as Virtual Functions (VF). | |
17 | Allocation of the VF can be dynamically controlled by the PF via | |
18 | registers encapsulated in the capability. By default, this feature is | |
19 | not enabled and the PF behaves as traditional PCIe device. Once it's | |
20 | turned on, each VF's PCI configuration space can be accessed by its own | |
21 | Bus, Device and Function Number (Routing ID). And each VF also has PCI | |
22 | Memory Space, which is used to map its register set. VF device driver | |
23 | operates on the register set so it can be functional and appear as a | |
24 | real existing PCI device. | |
25 | ||
26 | 2. User Guide | |
27 | ||
28 | 2.1 How can I enable SR-IOV capability | |
29 | ||
2597ba76 DD |
30 | Multiple methods are available for SR-IOV enablement. |
31 | In the first method, the device driver (PF driver) will control the | |
32 | enabling and disabling of the capability via API provided by SR-IOV core. | |
33 | If the hardware has SR-IOV capability, loading its PF driver would | |
34 | enable it and all VFs associated with the PF. Some PF drivers require | |
35 | a module parameter to be set to determine the number of VFs to enable. | |
36 | In the second method, a write to the sysfs file sriov_numvfs will | |
37 | enable and disable the VFs associated with a PCIe PF. This method | |
38 | enables per-PF, VF enable/disable values versus the first method, | |
39 | which applies to all PFs of the same device. Additionally, the | |
40 | PCI SRIOV core support ensures that enable/disable operations are | |
41 | valid to reduce duplication in multiple drivers for the same | |
42 | checks, e.g., check numvfs == 0 if enabling VFs, ensure | |
43 | numvfs <= totalvfs. | |
44 | The second method is the recommended method for new/future VF devices. | |
15b49bee YZ |
45 | |
46 | 2.2 How can I use the Virtual Functions | |
47 | ||
48 | The VF is treated as hot-plugged PCI devices in the kernel, so they | |
49 | should be able to work in the same way as real PCI devices. The VF | |
50 | requires device driver that is same as a normal PCI device's. | |
51 | ||
52 | 3. Developer Guide | |
53 | ||
54 | 3.1 SR-IOV API | |
55 | ||
56 | To enable SR-IOV capability: | |
2597ba76 | 57 | (a) For the first method, in the driver: |
15b49bee YZ |
58 | int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn); |
59 | 'nr_virtfn' is number of VFs to be enabled. | |
2597ba76 DD |
60 | (b) For the second method, from sysfs: |
61 | echo 'nr_virtfn' > \ | |
62 | /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs | |
15b49bee YZ |
63 | |
64 | To disable SR-IOV capability: | |
2597ba76 | 65 | (a) For the first method, in the driver: |
15b49bee | 66 | void pci_disable_sriov(struct pci_dev *dev); |
2597ba76 DD |
67 | (b) For the second method, from sysfs: |
68 | echo 0 > \ | |
69 | /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs | |
15b49bee | 70 | |
15b49bee YZ |
71 | 3.2 Usage example |
72 | ||
73 | Following piece of code illustrates the usage of the SR-IOV API. | |
74 | ||
63a29f74 | 75 | static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id) |
15b49bee YZ |
76 | { |
77 | pci_enable_sriov(dev, NR_VIRTFN); | |
78 | ||
79 | ... | |
80 | ||
81 | return 0; | |
82 | } | |
83 | ||
63a29f74 | 84 | static void dev_remove(struct pci_dev *dev) |
15b49bee YZ |
85 | { |
86 | pci_disable_sriov(dev); | |
87 | ||
88 | ... | |
89 | } | |
90 | ||
91 | static int dev_suspend(struct pci_dev *dev, pm_message_t state) | |
92 | { | |
93 | ... | |
94 | ||
95 | return 0; | |
96 | } | |
97 | ||
98 | static int dev_resume(struct pci_dev *dev) | |
99 | { | |
100 | ... | |
101 | ||
102 | return 0; | |
103 | } | |
104 | ||
105 | static void dev_shutdown(struct pci_dev *dev) | |
106 | { | |
107 | ... | |
108 | } | |
109 | ||
2597ba76 DD |
110 | static int dev_sriov_configure(struct pci_dev *dev, int numvfs) |
111 | { | |
112 | if (numvfs > 0) { | |
113 | ... | |
114 | pci_enable_sriov(dev, numvfs); | |
115 | ... | |
116 | return numvfs; | |
117 | } | |
118 | if (numvfs == 0) { | |
119 | .... | |
120 | pci_disable_sriov(dev); | |
121 | ... | |
122 | return 0; | |
123 | } | |
124 | } | |
125 | ||
15b49bee YZ |
126 | static struct pci_driver dev_driver = { |
127 | .name = "SR-IOV Physical Function driver", | |
128 | .id_table = dev_id_table, | |
129 | .probe = dev_probe, | |
63a29f74 | 130 | .remove = dev_remove, |
15b49bee YZ |
131 | .suspend = dev_suspend, |
132 | .resume = dev_resume, | |
133 | .shutdown = dev_shutdown, | |
2597ba76 | 134 | .sriov_configure = dev_sriov_configure, |
15b49bee | 135 | }; |