Features/XenPvops

paravirt_ops-based kernel-xen

Summary

Replacing the current forward-ported XenSource code in kernel-xen with the paravirt_ops based implementation from upstream. DomU support only, for now.

Xen pv_ops Dom0 support is being tracked on Features/XenPvopsDom0 .

Xen pv_ops domU support is now included in the vanilla/mainline Linux kernel, for x86_32, x86_64 and ia64, so separate kernel-xen package is dropped, and it is not anymore present in Fedora 10. Xen domU support is included in the normal kernel rpm. In Fedora 11, the kernel-PAE package is required for pv_ops support.

For upstream pv_ops kernel status see: http://wiki.xensource.com/xenwiki/XenParavirtOps

Owner

• Name: EduardoHabkost, MarkMcLoughlin

People involved: Stephen Tweedie, Chris Wright, Juan Quintela, Glauber Costa, Eduardo Habkost, DanielBerrange, MarkMcLoughlin et al.

Current status

• Targeted release: Fedora 9
• Last updated: 2008-04-07
• Percentage of completion: 100%
• 2009-01-24 update: Information on this page is very much outdated now, please see http://wiki.xensource.com/xenwiki/XenParavirtOps for upstream current status

Tracker bug for this feature: [1]

i386 domU

• Available in rawhide kernel-xen
• Core functionality merged upstream
• Supports blkfront / netfront
• Paravirt framebuffer
• Execshield enabled (bug #434759 )

x86_64 domU

• Available in rawhide kernel-xen
• x86_64 paravirt_ops will be in 2.6.25
• Doesn't include: SMP support, ia32 emulation
• Not yet ready for upstream submission

Detailed Description

Currently, the Fedora kernel-xen package is based on forward-porting of the XenSource patches from 2.6.18 to more recent kernel versions. This has many problems, including:

• XenSource code has no chance of being merged upstream, in the near future, making the forward-porting work needed for all new kernel versions.
• Lots of porting work for each new kernel version
• Because of the above, kernel-xen has been some releases behind the non-xen kernel package, and the lag between kernel and kernel-xen has been increasing constantly.

As of November 2007, the kernel-xen forward-porting was being finished for 2.6.22, and Linux 2.6.24 was about to be released. The effort needed for 2.6.23, 2.6.24 and later would have been even bigger with the introduction of paravirt_ops and the i386-x86_64 merge upstream. Thus, the decision was made to abandon the forward-porting effort and focus on upstream paravirt_ops.

See also this announcement:

http://www.redhat.com/archives/fedora-xen/2007-November/msg00106.html

and this update on Fedora 9 plans:

http://www.redhat.com/archives/fedora-xen/2008-March/msg00013.html

Alternatives

Alternatives that were considered (good ones and bad ones):

• Continue the current forward-porting approach.
• Consumes lots of work on forward-porting, forever.
• Will continue to lag many releases behind the non-xen kernel.
• It kills kittens.
• Dropping support for hosting Xen paravirt guests (and tell users to use KVM)
• Bad for existing users running Xen paravirt guests.
• Bad for new users because it requires hardware virtualization support.
• Support of hosting Xen paravirt guests using hardware virtualization only (e.g. Gerd's xenner, kvm-based)
• Requires hardware virtualization support.
• Still needs some work.
• Alternative approaches to support Xen paravirtualization, such as kvm-xen
• Probably would need more work than the xen paravirt_ops approach
• Other paravirt-based approaches not ABI compatible with existing Xen guests (e.g. kvm-lite):
• No good solution exists yet.

Benefit to Fedora

• Keep support for existing Xen guests running on Fedora hosts
• kernel-xen will be based on latest kernel available, and not an old release
• Hardware enablement since Xen will be running on latest LKML tree
• Merge upstream in the future, making the porting work not needed anymore to keep this feature
• After the merge upstream, it will be possible to support xen and non-xen on a single kernel binary

Scope

See the tracker bug for outstanding issues : [2]

Finished items:

• x86_64 paravirt_ops infrastructure in 2.6.25 (EduardoHabkost)
• x86_64 Xen DomU (EduardoHabkost)
• Done: hypercalls, fork & context switching, execve(), running user-space code, interrupt handling
• Disk and network I/O works
• Rawhide domU can be installed under Fedora 8 dom0
• Paravirt framebuffer for DomU (MarkusArmbruster)
• Frontend driver auto-loading (MarkMcLoughlin)
• Re-enable execshield; see bug #434759 (StephenTweedie)
• vmlinuz build (MarkMcLoughlin)

Test Plan

• Install a Fedora 8 host/Dom0 system
• Observe presence of Xen via /sys/hypervisor/
• Run 'xm list' and observe Domain-0
• Use virt-install to deploy a Fedora 9 paravirt guest

User Experience

• kernel-xen has same version as bare metal kernel

Dependencies

• x86_64 paravirt_ops in 2.6.25
• Updates to anaconda etc. to support hvc for console instead of xvc, detect presence of pvfb etc.

Contingency Plan

This whole feature is pretty high risk - it involves lots of scary work against the kernel. Already mitigating against some risk by continuing to target a separate kernel-xen RPM to minimize risk to the baremetal builds. There are several backup plans ...

Note: Option 2 is the current situation.

Option 1 (best of a bad lot)

This is a very likely scenario. Dom0 is not likely to be complete enough to risk adding as a patch to regular bare-metal in time for GA, so will keep as a separate RPM initially, cleaning up patch for F10

• Ship paravirt_ops enabled DomU for i386 & x86_64, implemented enough to get basic anaconda install working
• Ship basic Dom0 support ontop of pv_ops
• Keep both DomU and Dom0 in separate kernel-xen, albeit synced to kernel version and pv_ops enabled

Option 2 (semi-evil)

• Ship paravirt_ops enabled DomU only for i386 & x86_64, implemented enough to get basic anaconda install working
• Don't ship any Dom0 kernel - users can stay on F8 Dom0 until....
• ...ship a Dom0 kernel post-GA, since it isn't install time critical to have Dom0 Xen

Option 3 (semi-evil)

• Keep existing separate kernel-xen for Dom0 only based on Fedora 8 kernel-xen
• Paravirt ops DomU kernel part of bare metal 'kernel' RPM

Problematic (impossible) upgrade path, because there's no RPM dependancy which can obsolete kernel-xen for DomUs, but not obsolete it for Dom0.

Option 4 (very evil)

• Continue shipping existing forward ported kernel-xen 2.6.22 from Fedora 8 for DomU and Dom0.

This is basically total failure of our goals:-(

See also: 'Alternatives' heading earlier in this doc

Documentation

• None.

Release Notes

• kernel-xen does not have Dom0 support. Fedora 8 must continue to be used as the Xen host/Dom0.
• The Xen console is now "hvc" rather than "xvc"
• The Xen frontend network and disk frontend drivers have been renamed to xen-netfront and xen-blkfront from xennet and xenblk