Virtualization short cuts for noobs
Introduction
Purpose
Scope
Acronyms, descriptions
References
- http://www.linux-kvm.org/page/Documents
- KVM documentation page.
- http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge
- description of the linux bridge implementation.
- http://en.gentoo-wiki.com/wiki/Bridging_Network_Interfaces
- http://en.gentoo-wiki.com/wiki/KVM#Networking_2
- Shows direct and indirect bridging.
- http://blog.loftninjas.org/2008/08/18/using-attach-interface-with-libvirt-and-qemukvm-on-ubuntu/
- about attach-interface.
- http://tjworld.net/wiki/Linux/Ubuntu/VirtualMachinesWithVDENetworking
- Virtual Machines With VDE Networking
- http://www.mythicalbeast.co.uk/linux/dnsmasq_howto.html
- HOWTO for dnsmasq
- http://www.tuxradar.com/content/howto-linux-and-windows-virtualization-kvm-and-qemu
- hands-on KVM usage and a bit of background information.
- http://www.ibm.com/developerworks/library/l-linuxvirt/index.html
- An overview of virtualization methods, architectures, and implementations. From 2006.
- Virtualization_Quick_Start
- Get going with KVM.
Open issues
- how do we prevent container mem swap
- It seems that KVM will allow a containers memory to be swapped. This must be prevented.
- How to pin to cpus permanently
- it seems that editing the /etc/libvirt/qemu/domu.xml doesn't work.
Operating Environment
Which packages you need in order to run virtualization.
yum install @virtualization
or
- yum install libvirt
- yum install qemu-kvm
- yum install python-virtinst
- yum install virt-manager
- yum install virt-viewer
- yum install virt-top
- service libvirtd restart
Storage
- File based
- LVM Based
- NFS based
- iSCSI
Network
Network configuration needs.
- Bridging
- Direct
- NAT
- NAT/Route
Get direct bridging
Looks better: [Guest Configuration on Fedora 12 with Netcf] Source: [KVM w/o dnsmasq on CentOS 5.2 X64].
- cd /etc/sysconfig/network-scripts
- mkdir bak
- cp -p ifcfg-eth0 bak
- virsh iface-dumpxml eth0 > bak/eth0.xml
- cp bak/eth0.xml bak/kvmbr0.xml
- vi bak/kvmbr0.xml
- virsh iface-define bak/kvmbr0.xml
- virsh iface-list
- service network restart
- virsh edit fc13_1
- brctl show
The following didn't cut it, it simply prevented the DomU in starting, due to the missing 'default' network...
- virsh net-destroy default
- virsh net-autostart --disable default
- virsh net-autostart default
bak/kvmbr0.xml
<interface type='bridge' name='virbr0'>
<start mode="onboot"/>
<protocol family="ipv4">
<dhcp/>
</protocol>
<bridge stp="on">
<interface type="ethernet" name="eth0">
<mac address='d8:d3:85:e1:29:68'/>
</interface>
</bridge>
</interface>
virsh net-dumpxml default
<network>
<name>default</name>
<uuid>4eb269bb-b949-49ae-95b1-6e281b634c47</uuid>
<forward mode='route'/>
<bridge name='virbr0' stp='on' delay='0' />
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.2' end='192.168.122.254' />
</dhcp>
</ip>
</network>
Storage creation
LVM creation
- Create a partition of type 8e (Linux LVM)
- pvcreate /dev/sda5
- vgcreate LX /dev/sda5
- lvcreate --size 10G --name vol01 LX
Resource allocation
KVM
CPU
Host pinning
You pin the host OS to a CPU set, by telling the kenerl which CPUs to keep away from. So if you want your host OS to only use cpu0 then you must provide the kernel parm: isolcpus=1-3. In case of an 8 core CPU it would be 1-7 etc.
Add it to the end of the kernel entry in '/boot/grub/menu.lst'
Xen
CPU
Domain U pinning
If you want a Domain to have exclusive access to a cpu then you must pin all other DomUs to a cpu set that excludes the exclusive cpu. e.g.
On a 4 CPU system: If you want Dom1 to have exclusive access to cpu1, then you do the following:
- Pin all other domains to cpuset 2-3.
- Pin Dom1 to cpu1.
- Pin the host domain to cpu0.
- Or you could have Dom1 on cpu1 and both host and the other domains on the 0,2-3 cpu set.
I/O
Number of devices
Administration
virsh
virsh help lists the supported commands.
See also [28. Managing guests with virsh]
Virsh operations commands of interest
- console
- get access to the console in the DomU.
- destroy
- as if you pulled the power cord, preferably use 'shutdown'.
- reboot
- reboot the domain.
- shutdown
- shutdown the domain.
- undefine
- remove all references to the domain. (Note; not sure if it is actually *all*).
- dumpxml
- list
- List the active domains. Use '--all' to include the the inactive containers to the list.
Virsh network commands of interest
- domifstat
- interface statistics.
- virsh domifstat z3 vif7.0
- attach-interface
- attach a network interface.
- virsh attach-interface z3 bridge eth1 --target eth1 --mac xx:xx:xx:xx:xx:xx
- net-edit
- For an description of the xml structure see: [XML format]
- net-list
- list networks.
virsh cpu commands of interest
- vcpuping <domain-id> <vcpuid> <cpulist>
- Pin a domains virtual cpu to a list of cpu's.
- virsh vcpupin z2 0 2
- virsh vcpuinfo z2
VCPU: 0 CPU: 2 State: idle CPU time: 9.8s CPU Affinity: --y-----
virt-install
virt-mananger
remote management
Access guest files
- virt-cat
- virt-cat Guest /var/log/messages # or /var/log/dmesg etc
- guestfish
- guestfish --ro -i -d Guest
- ><fs> less /var/log/messages
- guestfish -i -d Guest
- virt-edit
- virt-edit Guest /boot/grub/grub.conf
><fs> emacs /boot/grub/grub.conf
- virt-rescue
- virt-rescue Guest
><rescue> mount /dev/vg_foo/lv_root /sysroot ><rescue> mount /dev/vda1 /sysroot/boot ><rescue> vi /sysroot/boot/grub/grub.conf
Creating DomUs
Creating a CentOS 5.5 DomU
Xen: source:NFS, storage:LVM network:bridge;4
virt-install -p --name=z4 --ram=512 --vcpus=1 --disk path=/dev/loom/z4 --os-type=linux
--os-variant=rhel5 --location=nfs:192.168.42.132:/exports/isos/centos55 -x
"ks=nfs:192.168.42.132:/exports/ks/anaconda-ks.cfg ip=192.168.42.31 netmask=255.255.255.0
gateway=192.168.42.132 dns=192.168.42.32"
KVM: source:NFS, storage:LVM network:bridged-4
virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --network=bridge:virbr0 --network=bridge:virbr1 --network=bridge:virbr2 --network=bridge:virbr3 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/kvm_centos55.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32"
Trouble shooting
virsh ops
error: Timed out during operation: cannot acquire state change lock
# virsh destroy z02 error: Failed to destroy domain z02 error: Timed out during operation: cannot acquire state change lock
- The system had just been rebooted.
- The z01 and z02 was active, when the reboot command was executed on the host.
- when the host came back up, both z01 and z02 was paused.
- virt-top, cleared the screen, and then didn't show anything.
- I could stop z01 with virsh destroy z01
- There was a qemu-kvm process running, and doing a kill on that finished off z02
virt-install
error: unable to connect to '/var/run/libvirt/libvirt-sock', libvirtd may need to be started: No such file or directory
run service libvirtd restart
ERROR Host does not support virtualization type 'xen'
running on KVM, the '-p | --para-virt' is not supported.
error: Failed to get local hostname
- The hostname must be set.
- The system must be able to do a dns lookup of the hostname.
- e.g. put the hostname in /etc/hosts
- Re: et-mgmt-tools virsh failed to get local hostname
virt-install --connect=qemu:///system --name=z01 --ram=512 --vcpus=1 --disk path=/dev/LX/vol01 --os-type=linux --os-variant=rhel5 --location=nfs:192.168.42.132:/exports/isos/centos55 -x "ks=nfs:192.168.42.132:/exports/ks/anaconda-ks.cfg ip=192.168.42.31 netmask=255.255.255.0 gateway=192.168.42.132 dns=192.168.42.32" Starting install... Retrieving file .treeinfo... | 834 B 00:00 ... Retrieving file vmlinuz... | 3.7 MB 00:00 ... Retrieving file initrd.img... | 15 MB 00:00 ... Creating domain... | 0 B 00:00 error: Failed to get local hostname Domain installation still in progress. You can reconnect to the console to complete the installation process.
Kickstart issues
That directory could not be mounted from the server
A1: Wrong IP address used for NFS server. A2: The path was wrong.
- tail /var/log/messages:
- refused mount request from 192.168.42.31 for /export/isos/centos55 (/): no export entry