Latest revision as of 03:23, 4 February 2010

KVM Stable PCI Addresses

Summary

Allow devices in KVM guest virtual machines to retain the same PCI address allocations as other devices are added or removed from the guest configuration.

This is particularily important for Windows guests in order to prevent warnings or reactivation when device addresses change.

Owner

Name: Markus Armbruster

Current status

Targeted release: Fedora 13
Last updated: 2010-01-26
Percentage of completion: 100%

Completed

New QEMU option -device permits specification of PCI device * address. It doesn't work for VGA devices, yet, but QEMU always assigns PCI device address 2 to the VGA.
libvirt uses this to keep PCI device addresses stable.

Detailed Description

QEMU allocates PCI addresses to devices (mostly) in the order it creates the devices, which depends in part on the order they appear on the command line. Built-in PCI devices - like the IDE, USB and VGA controllers - are allocated first.

Windows will warn users when a device's PCI address is changed and may even require the Windows install to be reactivated. In order to prevent this, we should do what we can to ensure that the device does not move PCI between slots as devices are added or removed to/from the guest configuration.

A related problem is that of guest ABI changes between versions of QEMU. That is, updating to a newer version of QEMU may cause devices to change subtly (e.g. PCI class of a device or additional capabilities) which again may require Windows to be reactivated. See also Features/KVM Stable Guest ABI.

A number of solutions were discussed upstream. In the end, it was decided to let QEMU allocate device addresses for new devices, then store them in the guest's XML configuration, and specify them on the command line when recreating the guest. This relies on an an improved info pci for querying the address, and the new -device for specifying it.

Benefit to Fedora

This feature would remove a significant issue with Fedora's virtualization support of Windows.

Scope

Requires work on both QEMU and libvirt. No other Fedora packages would be affected.

How To Test

Start a guest with two PCI NICs
Note the PCI slot numbers of each NIC
Remove the first NIC and re-start the guest
Check that the slot number of the second NIC hasn't changed

User Experience

It should be possible to add/remove devices without causing Windows guests to require reactivation.

Dependencies

The main dependency is a mature -device in upstream QEMU.

Contingency Plan

No contingency plan required; if one of the solutions is not implemented, the feature will not be available and nothing else will be affected.

Documentation

Izik's patch (November 2007)
Markus's patch series (Jan 2009)
Markus's machine description format RFC (February 2009)
Markus's tenth iteration of his machine description patches (April 2009)
First thread where saveabi idea came up (May 2009)
Most recent discussion (June 2009)

Release Notes

KVM guests in Fedora now have stable PCI addresses, reducing the chance that Windows guests will require reactivation as guest configuration is modified.

Comments and Discussion

See Talk:Features/KVM_Stable_PCI_Addresses

@@ Line 13: / Line 13: @@
 == Current status ==
-* Targeted release: [[Releases/12|Fedora 12]]
+* Targeted release: [[Releases/13|Fedora 13]]
-* Last updated: 2009-06-15
+* Last updated: 2010-01-26
-* Percentage of completion: 0%
+* Percentage of completion: 100%
-=== TODO ===
-* Settle on one of the proposed solutions upstream
-* Implement the qemu side
-* Implement the libvirt side
-* Testing
 === Completed ===
-* None
+* New QEMU option -device permits specification of PCI device * address.  It doesn't work for VGA devices, yet, but QEMU always assigns PCI device address 2 to the VGA.
+* libvirt uses this to keep PCI device addresses stable.
 == Detailed Description ==
-QEMU allocates PCI addresses to devices (roughly) in the order devices are supplied on the command line. Built-in PCI devices - like the IDE, USB and VGA controllers - are allocated first.
+QEMU allocates PCI addresses to devices (mostly) in the order it creates the devices, which depends in part on the order they appear on the command line. Built-in PCI devices - like the IDE, USB and VGA controllers - are allocated first.
 Windows will warn users when a device's PCI address is changed and may even require the Windows install to be reactivated. In order to prevent this, we should do what we can to ensure that the device does not move PCI between slots as devices are added or removed to/from the guest configuration.
@@ Line 36: / Line 30: @@
 A related problem is that of guest ABI changes between versions of QEMU. That is, updating to a newer version of QEMU may cause devices to change subtly (e.g. PCI class of a device or additional capabilities) which again may require Windows to be reactivated. See also [[Features/KVM Stable Guest ABI]].
-A number of solutions are being discussed upstream.
+A number of [[KVM_Stable_PCI_Addresses_Design_Notes|solutions]] were discussed upstream. In the end, it was decided to let QEMU allocate device addresses for new devices, then store them in the guest's XML configuration, and specify them on the command line when recreating the guest.  This relies on an an improved <code>info pci</code> for querying the address, and the new <code>-device</code> for specifying it.
-=== Solution 1 :: pci_addr= Command Line Option ===
-A <code>pci_addr=</code> command line option would be added to QEMU allowing each device's PCI address can be explicitly specified.
-In order to supply these addresses on the QEMU command line, libvirt would initially start the guest with no addresses specified and use the <code>info pci</code> to determine what address was allocated to each device. Those addresses would then be stored in the guest's XML configuration and be supplied on the command line when the guest is subsequently re-started. It is likely the <code>info pci</code> output would have to be improved to allow libvirt to match the devices it requested with the devices in the output.
-Pros:
-# Simplest option from QEMU's point of view
-# The libvirt changes would be relatively straightforward
-# Possible for this to be in place for Fedora 12
-Cons:
-# <code>info pci</code> needs to be made parseable
-# Only solves one aspect of the guest ABI problem
-=== Solution 2 :: pci_add Monitor Command ===
-Same as the first solution, except using the <code>pci_add</code> monitor command to specify PCI addresses instead of a <code>pci_addr=</code> command line argument.
-The idea is that qemu would be started using the <code>-S</code> option and the required devices added using <code>pci_add</code> before issuing the <code>continue</code> command.
-The issue of how libvirt would parse the output of <code>info pci</code> would remain.
-Pros:
-# Requires minimal changes upstream - probably just making <code>info pci</code> easier to parse
-# Possible for this to be in place for Fedora 12
-Cons:
-# Solution only under consideration because other solutions might not be agreed upstream in time
-# <code>info pci</code> needs to be made parseable
-# libvirt would have to learn this new method of starting guests
-# Only solves one aspect of the guest ABI problem
-=== Solution 3 :: Machine Description File ===
-A machine description file containing a device tree could be supplied to QEMU which, amongst other things, would specify the PCI addresses of devices.
-libvirt would initially start the guest with a minimal (i.e. without PCI addresses) machine description file and then use a monitor command to obtain the entire tree (i.e. with PCI addresses). The dumped tree would then be retained (probably embedded in the guest XML config) and used when the guest is re-started in future.
-In order to use this machine description file, libvirt would need to be able to parse and modify the full tree obtained from the monitor command. For example, if a device was removed from the guest configuration, libvirt would need to remove it from the machine description before re-starting the guest.
-What may not be obvious here is that libvirt currently does not need to detect when guest configuration has changed. It merely maps the guest configuration to a qemu command line. If it switches to supplying a machine description file, it would need to iterate over the guest XML and add, remove or modify devices in the device tree.
-Pros:
-# Solves the guest ABI issue more generally
-# Progress is being made upstream in QEMU towards this solution
-Cons:
-# Significant work still required in QEMU
-# Very complex solution for libvirt
-# Highly unlikely to be in place for Fedora 12
-=== Solution 4 :: Compat Hints File ===
-A monitor command (e.g. <code>saveabi</code> or <code>savevm --abi</code>) would be added which would export a device tree containing enough information for QEMU to configure devices in exactly the same way as when the tree was exported. A simple implementation might just contain a version number and PCI address for each device.
-The exported file would be opaque from the point of view of qemu users. It would merely serve as compat hints which augment the supplied guest configuration. For example, if a device was removed from the guest configuration but remained in the compat hints file, then QEMU would merely ignore the hints for that device.
-libvirt would export a new compat hints file each time it starts a QEMU guest or hotplugs a device. The compat hints would then be stored (probably embedded in the guest XML configuration) and be passed to QEMU via the command line when the guest is re-started.
-Pros:
-# Solves the guest ABI issue more generally
-# Straightforward solution for libvirt
-Cons:
-# Upstream not keen on adding both compat hints and machine description formats
-# Quite unlikely to be ready for Fedora 12
 == Benefit to Fedora ==
@@ Line 120: / Line 38: @@
 == Scope ==
-Each of the proposed solutions requires significant work in both QEMU and libvirt. No other Fedora packages would be affected.
+Requires work on both QEMU and libvirt. No other Fedora packages would be affected.
 == How To Test ==
@@ Line 131: / Line 49: @@
 == User Experience ==
-It should be possible to add/remove devices without causing Window's guests to require reactivation.
+It should be possible to add/remove devices without causing Windows guests to require reactivation.
 == Dependencies ==
-The main dependency is agreeing upstream on the QEMU part of the solution.
+The main dependency is a mature <code>-device</code> in upstream QEMU.
 == Contingency Plan ==
@@ Line 152: / Line 70: @@
 == Release Notes ==
-KVM guests in Fedora 12 now have stable PCI addresses, reducing the chance that Windows guests will require reactivation as guest configuration is modified.
+KVM guests in Fedora now have stable PCI addresses, reducing the chance that Windows guests will require reactivation as guest configuration is modified.
 == Comments and Discussion ==
-* See [[Talk:Features/KVM Stable PCI Addresses]]
+* See [[Talk:Features/KVM_Stable_PCI_Addresses]]
+[[Category:FeatureAcceptedF13]]
-[[Category:Virtualization|KVM Stable PCI Addresses]]
-[[Category:FeaturePageIncomplete]]
-<!-- Category:FeatureReadyForWrangler -->
-[[Category:F12_Virt_Features|KVM Stable PCI Addresses]]
+[[Category:Virtualization|KVM Stable PCI Addresses]]
+[[Category:F13_Virt_Features|KVM Stable PCI Addresses]]

Search

Features/KVM Stable PCI Addresses: Difference between revisions