SR-IOV
Summary
Single Root I/O Virtualization (SR-IOV) is a PCI feature which allows virtual functions (VF) to be created that share the resources of a physical function (PF).
SR-IOV is primarily useful in allowing a single PCI device to be shared amongst multiple virtual machines.
Owner
- Name: Chris Wright
Current status
- Targeted release: Fedora 12
- Last updated: 2009-08-05
- Percentage of completion: 100%
TODO
- Test that VF device assignment works with Fedora virt tools
Completed
- Get PCI_IOV support upstream (included in v2.6.30)
Detailed Description
SR-IOV allows a single PCI device to be shared amongst multiple virtual machines while retaining the performance benefit of assigning a PCI device to a virtual machine. A common example is where a single SR-IOV capable NIC - with perhaps only a single physical network port - might be shared with multiple virtual machines by assigning a virtual function to each VM.
SR-IOV support is implemented in the kernel. The core implementation is contained in the PCI subsystem, but there must also be driver support for both the Physical Function (PF) and Virtual Function (VF) devices. With an SR-IOV capable device one can allocate VFs from a PF. The VFs surface as PCI devices which are backed on the physical PCI device by resources (queues, and register sets).
In 2.6.30, the Intel® 82576 Gigabit Ethernet Controller is a supported. The igb driver has PF support and the igbvf has VF support.
In 2.6.31 the Neterion® X3100™ is supported. This device uses the same vxge driver for the PF as well as the VFs.
Benefit to Fedora
SR-IOV support further extends Fedora's ability to act as a high-performance server Virtualization solution. With the F11 feature PCI device assignment Fedora can assign a physical PCI device to a guest. With SR-IOV support Fedora can now allocate resources from a PCI device which itself supports SR-IOV. These resources, called Virtual Functions, surface as new PCI devices which can be assigned to a guest OS. This gives the same performance benefit of assigning a physical PCI device to a guest, however, it eliminates the need to have the server filled with physical PCI devices (which may not be physically possible for large counts of PCI devices), and the administrative complications of managing a large number of devices (these are typically NICs or HBAs, so there's cabling involved). Because the guest OS is effectively driving the hardware directly, the I/O performance is on par with bare metal performance. A definite improvement over emulated I/O devices or paravirtual I/O devices.
Scope
The scope of this feature is entirely limited to the kernel.
The PCI device assignment feature in Fedora 11 is required for SR-IOV to be useful.
How To Test
- Ensure that you have a supported SR-IOV capable device (currently only 82576 NIC)
- Ensure the PF driver allocates a number of VFs (e.g.
modprobe igb max_vfs=7) - Assign the VF to a guest (see Features/KVM PCI Device Assignment)
- Load the VF driver in the guest and ensure the device works as expected (igbvg is the VF driver for 82576)
User Experience
As above, users should be able to allocate VFs and assign them to guest virtual machines, allowing the physical resources of the PF to be shared with multiple guests.
Dependencies
None.
Contingency Plan
None required.
Documentation
- Single Root I/O Virtualization and Sharing Specification, Revision 1.0. Available from PCI SIG.
- Intel® 82576 Gigabit Ethernet Controller
Release Notes
Fedora 12 adds support for PCI Single Root I/O Virtualization (SR-IOV) which allows capable hardware to be shared with multiple guest virtual machines.