From Fedora Project Wiki

Virtualization short cuts for noobs

Introduction

Purpose

Scope

Acronyms, descriptions

References

http://www.linux-kvm.org/page/Documents
KVM documentation page.
http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge
description of the linux bridge implementation.
http://en.gentoo-wiki.com/wiki/Bridging_Network_Interfaces
http://en.gentoo-wiki.com/wiki/KVM#Networking_2
Shows direct and indirect bridging.
http://blog.loftninjas.org/2008/08/18/using-attach-interface-with-libvirt-and-qemukvm-on-ubuntu/
about attach-interface.
http://tjworld.net/wiki/Linux/Ubuntu/VirtualMachinesWithVDENetworking
Virtual Machines With VDE Networking
http://www.mythicalbeast.co.uk/linux/dnsmasq_howto.html
HOWTO for dnsmasq
http://www.tuxradar.com/content/howto-linux-and-windows-virtualization-kvm-and-qemu
hands-on KVM usage and a bit of background information.
http://www.ibm.com/developerworks/library/l-linuxvirt/index.html
An overview of virtualization methods, architectures, and implementations. From 2006.
Virtualization_Quick_Start
Get going with KVM.

Open issues

how do we prevent container mem swap
It seems that KVM will allow a containers memory to be swapped. This must be prevented.
How to pin to cpus permanently
it seems that editing the /etc/libvirt/qemu/domu.xml doesn't work.

Operating Environment

Which packages you need in order to run virtualization.

yum install @virtualization

or

  1. yum install libvirt
  2. yum install qemu-kvm
  3. yum install python-virtinst
  4. yum install virt-manager
  5. yum install virt-viewer
  6. yum install virt-top
  7. service libvirtd restart

Storage

  • File based
  • LVM Based
  • NFS based
  • iSCSI

Network

Network configuration needs.

  • Bridging
    • Direct
    • NAT
  • NAT/Route


Get direct bridging

Looks better: [Guest Configuration on Fedora 12 with Netcf] Source: [KVM w/o dnsmasq on CentOS 5.2 X64].

  1. cd /etc/sysconfig/network-scripts
  2. mkdir bak
  3. cp -p ifcfg-eth0 bak
  4. virsh iface-dumpxml eth0 > bak/eth0.xml
  5. cp bak/eth0.xml bak/kvmbr0.xml
  6. vi bak/kvmbr0.xml
  7. virsh iface-define bak/kvmbr0.xml
  8. virsh iface-list
  9. service network restart
  10. virsh edit fc13_1
  11. brctl show


The following didn't cut it, it simply prevented the DomU in starting, due to the missing 'default' network...

  1. virsh net-destroy default
  2. virsh net-autostart --disable default


  1. virsh net-autostart default

bak/kvmbr0.xml

<interface type='bridge' name='virbr0'>
  <start mode="onboot"/>
  <protocol family="ipv4">
    <dhcp/>
  </protocol>
  <bridge stp="on">
    <interface type="ethernet" name="eth0">
      <mac address='d8:d3:85:e1:29:68'/>
    </interface>
  </bridge>
</interface>


virsh net-dumpxml default

<network>
  <name>default</name>
  <uuid>4eb269bb-b949-49ae-95b1-6e281b634c47</uuid>
  <forward mode='route'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
</network>

Storage creation

LVM creation

  1. Create a partition of type 8e (Linux LVM)
  2. pvcreate /dev/sda5
  3. vgcreate LX /dev/sda5
  4. lvcreate --size 10G --name vol01 LX

Resource allocation

KVM

CPU

Host pinning

You pin the host OS to a CPU set, by telling the kenerl which CPUs to keep away from. So if you want your host OS to only use cpu0 then you must provide the kernel parm: isolcpus=1-3. In case of an 8 core CPU it would be 1-7 etc.

Add it to the end of the kernel entry in '/boot/grub/menu.lst'

Xen

CPU


Domain U pinning
Pinning does not mean exclusive access, if you pin Dom1 to cpu1 then Dom1 can only run on cpu1, but it does not prevent other domains to run on cpu1.

If you want a Domain to have exclusive access to a cpu then you must pin all other DomUs to a cpu set that excludes the exclusive cpu. e.g.

 On a 4 CPU system:
If you want Dom1 to have exclusive access to cpu1, then you do the following:
  1. Pin all other domains to cpuset 2-3.
  2. Pin Dom1 to cpu1.
  3. Pin the host domain to cpu0.
    • Or you could have Dom1 on cpu1 and both host and the other domains on the 0,2-3 cpu set.

I/O

Number of devices

Administration

virsh

virsh help lists the supported commands.

See also [28. Managing guests with virsh]

Virsh operations commands of interest

console
get access to the console in the DomU.
destroy
as if you pulled the power cord, preferably use 'shutdown'.
reboot
reboot the domain.
shutdown
shutdown the domain.
undefine
remove all references to the domain. (Note; not sure if it is actually *all*).


dumpxml
list
List the active domains. Use '--all' to include the the inactive containers to the list.

Virsh network commands of interest

domifstat
interface statistics.
  • virsh domifstat z3 vif7.0
attach-interface
attach a network interface.
  • virsh attach-interface z3 bridge eth1 --target eth1 --mac xx:xx:xx:xx:xx:xx
net-edit
For an description of the xml structure see: [XML format]
net-list
list networks.

virsh cpu commands of interest

vcpuping <domain-id> <vcpuid> <cpulist>
Pin a domains virtual cpu to a list of cpu's.
    • virsh vcpupin z2 0 2
  1. virsh vcpuinfo z2

VCPU: 0 CPU: 2 State: idle CPU time: 9.8s CPU Affinity: --y-----

virt-install

virt-mananger

remote management

Access guest files

virt-cat
virt-cat Guest /var/log/messages # or /var/log/dmesg etc
guestfish
guestfish --ro -i -d Guest
    • ><fs> less /var/log/messages
    • guestfish -i -d Guest
virt-edit
virt-edit Guest /boot/grub/grub.conf
><fs> emacs /boot/grub/grub.conf
virt-rescue
virt-rescue Guest
><rescue> mount /dev/vg_foo/lv_root /sysroot
><rescue> mount /dev/vda1 /sysroot/boot
><rescue> vi /sysroot/boot/grub/grub.conf

Creating DomUs

Creating a CentOS 5.5 DomU

Xen: source:NFS, storage:LVM network:bridge;4

virt-install -p --name=z4 --ram=512 --vcpus=1 --disk path=/dev/loom/z4 --os-type=linux --os-variant=rhel5 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/anaconda-ks.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32"


KVM: source:NFS, storage:LVM network:bridged-4

virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --network=bridge:virbr0 --network=bridge:virbr1 --network=bridge:virbr2 --network=bridge:virbr3 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/kvm_centos55.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32"

Trouble shooting

virsh ops

error: Timed out during operation: cannot acquire state change lock

# virsh destroy z02
error: Failed to destroy domain z02
error: Timed out during operation: cannot acquire state change lock
  • The system had just been rebooted.
    • The z01 and z02 was active, when the reboot command was executed on the host.
  • when the host came back up, both z01 and z02 was paused.
  • virt-top, cleared the screen, and then didn't show anything.
  • I could stop z01 with virsh destroy z01
  • There was a qemu-kvm process running, and doing a kill on that finished off z02

See also: http://us.generation-nt.com/answer/bug-602715-libvirt-bin-locking-problem-libvirt-help-201086371.html

virt-install

error: unable to connect to '/var/run/libvirt/libvirt-sock', libvirtd may need to be started: No such file or directory

run service libvirtd restart


ERROR Host does not support virtualization type 'xen'

running on KVM, the '-p | --para-virt' is not supported.


error: Failed to get local hostname

virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/anaconda-ks.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32"


Starting install...
Retrieving file .treeinfo...                                                     |  834 B     00:00 ... 
Retrieving file vmlinuz...                                                       | 3.7 MB     00:00 ... 
Retrieving file initrd.img...                                                    |  15 MB     00:00 ... 
Creating domain...                                                               |    0 B     00:00     
error: Failed to get local hostname

Domain installation still in progress. You can reconnect to 
the console to complete the installation process.

Kickstart issues

That directory could not be mounted from the server

A1: Wrong IP address used for NFS server. A2: The path was wrong.

  • tail /var/log/messages:
    • refused mount request from 192.168.42.31 for /export/isos/centos55 (/): no export entry