Latest revision as of 18:09, 4 February 2013

Virtio-Serial

Summary

This feature modifies the current single-port virtio-console device to guests running on top of qemu and kvm. It exposes multiple ports to the guest in the form of simple char devices for simple IO between the guest and host userspaces. It also allows for multiple such devices to be exposed, lifting the current single device restriction.

Owner

Name: Amit Shah
email: amit.shah@redhat.com

Current status

Targeted release: Fedora 13
Last updated: 2010-02-08
Percentage of completion: 100%

Detailed Description

Also known as 'vmchannel', a transport mechanism is needed for communication between the host userspace and guest userspace for achieving things like making clipboard copy/paste work seamlessly across the host and guest, locking the guest screen in case the vnc session to the guest is closed and so on. This can be used in offline cases as well, for example with libguestfs to probe which file systems the guest uses, the apps installed, etc.

Virtio-serial is just the transport protocol that will enable such applications to be written. It has two parts: (a) device emulation in qemu that presents a virtio-pci device to the guest and (b) a guest driver that presents a char device interface to userspace applications.

Benefit to Fedora

Some benefits include:

a communication channel between the guest and the host even when guest networking is disabled
applications can be written on top of the transport to achieve better guest-host cooperation, eg., clipboard copy/paste support between the host and the guest
libguestfs currently uses a low-performing vmchannel interface. Updating libguestfs to use this interface will achieve better speeds and flexibility.
SPICE will be able to use this infrastructure.
Anaconda logs could be sent to the host for messages that are emitted before networking is setup.

Scope

Changes are required in QEMU and Linux. A few udev rules will have to be added to dynamically create ports for the device and specify default permissions.

How To Test

There are quite a few ways to test the functionality:

Spawn a virtioconsole with new guest kernel and qemu images. Work on the console.
Spawn multiple virtioconsoles.
Transfer files from the host to the guest (and vice-versa) over virtio-serial ports and compare sha1sums of the files on the source and destination
Try all of the above at the same time
Try the above with different guest smp options
Try the testsuite at the link below for exercising various open/read/write/poll paths that test for various options

Git repository for testsuite:

http://fedorapeople.org/cgit/amitshah/public_git/test-virtserial.git/

When starting qemu or kvm guests from the command line, add something like this:

-device virtio-serial \
-chardev socket,path=/tmp/foo,server,nowait,id=foo \
-device virtserialport,chardev=foo,name=org.fedoraproject.port.0

This will create a device and expose one serial port to the guest. The guest kernel should load the virtio_console.ko module and expose a /dev/vport0p1 port to guest userspace app developers. A sysfs attribute is placed in /sys/class/virtio-ports/vport0p1/name that contains the text "org.fedoraproject.port.0". A udev rule will create a symlink in /dev/virtio-ports:

/dev/virtio-ports/org.fedoraproject.port.0 -> /dev/vport0p1

Data written into host's /tmp/foo (via a utility like socat) will then be relayed to the guest and a guest app should be able to read the data from /dev/vport0p1 or /dev/virtio-ports/org.fedoraproject.port.0.

/dev/vportNp0 is reserved for the first virtio console.

Testing console IO

Create virtio-console ports by:

-device virtio-serial
-chardev socket,path=/tmp/foo,server,nowait,id=foo \
-chardev socket,path=/tmp/bar,server,nowait,id=bar \
-device virtioconsole,chardev=foo,name=org.fedoraproject.console.foo
-device virtioconsole,chardev=bar,name=org.fedoraproject.console.bar

The console ports will be available in the guest as /dev/hvc0 and /dev/hvc1.

Spawn console ports via:

# agetty /dev/hvc0 9600 vt100
# agetty /dev/hvc1 9600 vt100

On the host:

$ socat /tmp/foo -
$ socat /tmp/bar -

Will get the two console terminals.

Invocation Options Available

Device

The virtio-serial device itself has two options:

max_ports: This option can be used to set the maximum number of allowed ports for this device. The default is 31 (the current maximum). If more ports are needed, two such devices can be spawned. If it's expected that the number of ports ever needed across the lifetime of the guest will be lower than the default, specifying that value here will result in some resource savings (memory, MSI vectors). Example:

-device virtio-serial,max_ports=2

vectors: This option specifies the number of MSI vectors to be used for the device. It only makes sense for architectures that use the PCI bus. The default is to use as many vectors as there are ports (plus 1 for internal usage). If '0' is specified here, MSI will be disabled and a GSI interrupt will be used. Example:

-device virtio-serial,vectors=4
(or)
-device virtio-serial-pci,vectors=4

The default qdev options are available as well. For example, to create two virtio-serial devices and to differentiate between the two, something like the following will work:

-device virtio-serial,id=ser0,max_ports=2 -device virtio-serial,id=ser1,max_ports=3 \
-device virtserialport,bus=ser0.0,chardev=portchar0,id=port0,name=org.fedoraproject.port.0 \
-device virtioconsole,bus=ser1.0,chardev=conschar0,id=cons0,name=org.fedoraproject.console.0

Ports

Individual ports have port-specific options available:

nr: This option specifies the port number the port is to be given for enumeration within qemu. This option is useful when using hot-plug/unplug operations and migration. If this parameter is not specified, qemu will allocate a number to the port.

name: This option gives a 'name' to the port, which is exposed in the guest via sysfs and a symlink is created to the actual port, as described above.

The virtserialport and virtconsole ports have an extra parameter:

chardev: This parameter specifies the host-side qemu chardev the port is associated with for IO.

Hot-plug / Hot-unplug

Using the qemu monitor, ports can be hot-plugged as well as unplugged:

(qemu) device_add virtserialport,name=org.fedoraproject.port.0,id=port0
(qemu) device_del port0

API

A page describing the Linux and Windows Guest APIs and host qemu APIs along with some gotchas is put up at virtio-serial-API.

User Experience

Virtio-serial is just the transport and by itself won't be user-visible. Applications written on top of virtio-serial to communicate data between the host and guest OSes will bring user-visible changes.

Dependencies

The QEMU patches are accepted upstream and will be part of the next major release (post-0.12). The kernel patches are in Rusty's queue and will be part of the 2.6.35 release. A udev rule that creates the symlinks for port names is included in udev release 151 (with a bugfix, that should be part of 152).

Contingency Plan

QEMU currently contains an alternative implementation of vmchannel using userland networking which is slower but similar in feature set. Continue using that interface in case virtio-serial is not ready.

Documentation

http://www.linux-kvm.org/page/VMchannel_Requirements

http://log.amitshah.net/2010/09/communication-between-guests-and-hosts/

http://www.linux-kvm.org/page/Virtio-serial_API

Release Notes

The virtio-console pci device is now equipped to handle multiple console ports as well as generic ports for guests running on top of qemu and kvm. This facilitates simple guest <-> host communication.

Comments and Discussion

See Talk:VirtioSerial

@@ Line 1: / Line 1: @@
-<!-- The actual name of your feature page should look something like: Features/YourFeatureName.  This keeps all features in the same namespace -->
+= Virtio-Serial =
-= Feature Name =
-Features/Virtio-Serial
 == Summary ==
-<!-- A sentence or two summarizing what this feature is and what it will do.  This information is used for the overall feature summary page for each release. -->
+This feature modifies the current single-port virtio-console device to guests running on top of qemu and kvm. It exposes multiple ports to the guest in the form of simple char devices for simple IO between the guest and host userspaces. It also allows for multiple such devices to be exposed, lifting the current single device restriction.
-This feature adds a virtio-serial device to guests running on top of qemu and kvm. It exposes multiple ports to the guest in the form of simple char devices for simple IO between the guest and host userspaces.
 == Owner ==
-<!--This should link to your home wiki page so we know who you are-->
 * Name: [[User:Amitshah| Amit Shah]]
-<!-- Include you email address that you can be reached should people want to contact you about helping with your feature, status is requested, or  technical issues need to be resolved-->
 * email: amit.shah@redhat.com
 == Current status ==
-* Targeted release: [[Releases/12|Fedora 12]]
+* Targeted release: [[Releases/13|Fedora 13]]
-* Last updated: 2009-06-29
+* Last updated: 2010-02-08
-* Percentage of completion: 60%
+* Percentage of completion: 100%
-<!-- CHANGE THE "FedoraVersion" TEMPLATES ABOVE TO PLAIN NUMBERS WHEN YOU COMPLETE YOUR PAGE. -->
 == Detailed Description ==
-<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
 Also known as 'vmchannel', a transport mechanism is needed for communication between the host userspace and guest userspace for achieving things like making clipboard copy/paste work seamlessly across the host and guest, locking the guest screen in case the vnc session to the guest is closed and so on. This can be used in offline cases as well, for example with libguestfs to probe which file systems the guest uses, the apps installed, etc.
@@ Line 29: / Line 19: @@
 == Benefit to Fedora ==
-<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new feature, what capabilities does it bring? Why will Fedora become a better distribution or project because of this feature?-->
 Some benefits include:
 * a communication channel between the guest and the host even when guest networking is disabled
 * applications can be written on top of the transport to achieve better guest-host cooperation, eg., clipboard copy/paste support between the host and the guest
-* libguestfs currently uses a low-performing vmchannel interface. Updating libguestfs to use this interface will achieve better speeds.
+* libguestfs currently uses a low-performing vmchannel interface. Updating libguestfs to use this interface will achieve better speeds and flexibility.
+* SPICE will be able to use this infrastructure.
+* Anaconda logs could be sent to the host for messages that are emitted before networking is setup.
 == Scope ==
-<!-- What work do the developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
+Changes are required in QEMU and Linux. A few udev rules will have to be added to dynamically create ports for the device and specify default permissions.
-Changes are required in QEMU and Linux. A few udev rules will have to be added to dynamically create ports for the device.
 == How To Test ==
-<!-- This does not need to be a full-fledged document.  Describe the dimensions of tests that this feature is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.
+There are quite a few ways to test the functionality:
+* Spawn a virtioconsole with new guest kernel and qemu images. Work on the console.
+* Spawn multiple virtioconsoles.
+* Transfer files from the host to the guest (and vice-versa) over virtio-serial ports and compare sha1sums of the files on the source and destination
+* Try all of the above at the same time
+* Try the above with different guest smp options
+* Try the testsuite at the link below for exercising various open/read/write/poll paths that test for various options
+Git repository for testsuite:
+http://fedorapeople.org/cgit/amitshah/public_git/test-virtserial.git/
+When starting qemu or kvm guests from the command line, add something like this:
+<pre>
+-device virtio-serial \
+-chardev socket,path=/tmp/foo,server,nowait,id=foo \
+-device virtserialport,chardev=foo,name=org.fedoraproject.port.0
+</pre>
-Remember that you are writing this how to for interested testers to use to check out your feature - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your feature.
+This will create a device and expose one serial port to the guest. The guest kernel should load the virtio_console.ko module and expose a /dev/vport0p1 port to guest userspace app developers. A sysfs attribute is placed in <code>/sys/class/virtio-ports/vport0p1/name</code> that contains the text "org.fedoraproject.port.0". A udev rule will create a symlink in <code>/dev/virtio-ports</code>:
-A good "how to test" should answer these four questions:
+<code>
+/dev/virtio-ports/org.fedoraproject.port.0 -> /dev/vport0p1
+</code>
-. What special hardware / data / etc. is needed (if any)?
+Data written into host's <code>/tmp/foo</code> (via a utility like socat) will then be relayed to the guest and a guest app should be able to read the data from <code>/dev/vport0p1</code> or <code>/dev/virtio-ports/org.fedoraproject.port.0</code>.
-. How do I prepare my system to test this feature? What packages
-need to be installed, config files edited, etc.?
-. What specific actions do I perform to check that the feature is
-working like it's supposed to?
-. What are the expected results of those actions?
--->
-When starting qemu or kvm guests from the command line, a new option, -virtioserial unix:/path/to/socket will create a PCI device and expose one serial port to the guest. The guest kernel should load the virtio-serial.ko module and expose a /dev/vmch0 port to guest userspace app developers. Data written into host's /path/to/socket will then be relayed to the guest and a guest app should be able to read the data from /dev/vmch0.
+<code>/dev/vportNp0</code> is reserved for the first virtio console.
+=== Testing console IO ===
+Create virtio-console ports by:
+<pre>
+-device virtio-serial
+-chardev socket,path=/tmp/foo,server,nowait,id=foo \
+-chardev socket,path=/tmp/bar,server,nowait,id=bar \
+-device virtioconsole,chardev=foo,name=org.fedoraproject.console.foo
+-device virtioconsole,chardev=bar,name=org.fedoraproject.console.bar
+</pre>
+The console ports will be available in the guest as <code>/dev/hvc0</code> and <code>/dev/hvc1</code>.
+Spawn console ports via:
+<pre>
+# agetty /dev/hvc0 9600 vt100
+# agetty /dev/hvc1 9600 vt100
+</pre>
+On the host:
+<pre>
+$ socat /tmp/foo -
+$ socat /tmp/bar -
+</pre>
+Will get the two console terminals.
+=== Invocation Options Available ===
+==== Device ====
+The virtio-serial device itself has two options:
+* <code>max_ports</code>: This option can be used to set the maximum number of allowed ports for this device. The default is 31 (the current maximum). If more ports are needed, two such devices can be spawned. If it's expected that the number of ports ever needed across the lifetime of the guest will be lower than the default, specifying that value here will result in some resource savings (memory, MSI vectors). Example:
+<pre>
+-device virtio-serial,max_ports=2
+</pre>
+* <code>vectors</code>: This option specifies the number of MSI vectors to be used for the device. It only makes sense for architectures that use the PCI bus. The default is to use as many vectors as there are ports (plus 1 for internal usage). If '0' is specified here, MSI will be disabled and a GSI interrupt will be used. Example:
+<pre>
+-device virtio-serial,vectors=4
+(or)
+-device virtio-serial-pci,vectors=4
+</pre>
+The default qdev options are available as well. For example, to create two virtio-serial devices and to differentiate between the two, something like the following will work:
+<pre>
+-device virtio-serial,id=ser0,max_ports=2 -device virtio-serial,id=ser1,max_ports=3 \
+-device virtserialport,bus=ser0.0,chardev=portchar0,id=port0,name=org.fedoraproject.port.0 \
+-device virtioconsole,bus=ser1.0,chardev=conschar0,id=cons0,name=org.fedoraproject.console.0
+</pre>
+==== Ports ====
+Individual ports have port-specific options available:
+* <code>nr</code>: This option specifies the port number the port is to be given for enumeration within qemu. This option is useful when using hot-plug/unplug operations and migration. If this parameter is not specified, qemu will allocate a number to the port.
+* <code>name</code>: This option gives a 'name' to the port, which is exposed in the guest via sysfs and a symlink is created to the actual port, as described above.
+The <code>virtserialport</code> and <code>virtconsole</code> ports have an extra parameter:
+* <code>chardev</code>: This parameter specifies the host-side qemu chardev the port is associated with for IO.
+=== Hot-plug / Hot-unplug ===
+Using the qemu monitor, ports can be hot-plugged as well as unplugged:
+<pre>
+(qemu) device_add virtserialport,name=org.fedoraproject.port.0,id=port0
+(qemu) device_del port0
+</pre>
+=== API ===
+A page describing the Linux and Windows Guest APIs and host qemu APIs along with some gotchas is put up at [http://www.linux-kvm.org/page/Virtio-serial_API virtio-serial-API].
 == User Experience ==
-<!-- If this feature is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
 Virtio-serial is just the transport and by itself won't be user-visible. Applications written on top of virtio-serial to communicate data between the host and guest OSes will bring user-visible changes.
 == Dependencies ==
-<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this feature depends?  In other words, completion of another feature owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel feature)? -->
+The QEMU patches are accepted upstream and will be part of the next major release (post-0.12). The kernel patches are in Rusty's queue and will be part of the 2.6.35 release. A udev rule that creates the symlinks for port names is included in udev release 151 (with a bugfix, that should be part of 152).
-This feature depends on acceptance of the patches by the QEMU and Linux kernel communities.
 == Contingency Plan ==
-<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "None necessary, revert to previous release behaviour."  Or it might not.  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
 QEMU currently contains an alternative implementation of vmchannel using userland networking which is slower but similar in feature set. Continue using that interface in case virtio-serial is not ready.
 == Documentation ==
-<!-- Is there upstream documentation on this feature, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
 http://www.linux-kvm.org/page/VMchannel_Requirements
+http://log.amitshah.net/2010/09/communication-between-guests-and-hosts/
+http://www.linux-kvm.org/page/Virtio-serial_API
 == Release Notes ==
-<!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
+* The virtio-console pci device is now equipped to handle multiple console ports as well as generic ports for guests running on top of qemu and kvm. This facilitates simple guest <-> host communication.
-<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this feature, indicate them here.  You can also link to upstream documentation if it satisfies this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release. -->
-* New virtio-serial pci device exposed to guests running on top of qemu and kvm. A single device exposes multiple serial ports for simple guest <-> host communication.
 == Comments and Discussion ==
-* See [[Talk:Features/VirtioSerial]]  <!-- This adds a link to the "discussion" tab associated with your page.  This provides the ability to have ongoing comments or conversation without bogging down the main feature page -->
+* See [[Talk:VirtioSerial]]
-[[Category:FeaturePageIncomplete]]
-<!-- When your feature page is completed and ready for review -->
-<!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler -->
-<!-- After review, the feature wrangler will move your page to Category:FeatureReadyForFesco... if it still needs more work it will move back to Category:FeaturePageIncomplete-->
-<!-- A pretty picture of the page category usage is at: https://fedoraproject.org/wiki/Features/Policy/Process -->
-<!-- Note that the current Feature guidelines require useful Scope and Test Plans at certain milestones; QA is responsible for checking these, and will change this category as needed. -->
+[[Category:FeatureAcceptedF13]]
+[[Category:Virtualization|VirtioSerial]]
+[[Category:F13_Virt_Features|VirtioSerial]]

Search

Features/VirtioSerial: Difference between revisions