Cloud image generation short cuts for noobs

Cloud image generation short cuts for noobs

Introduction

References

• http://linux.die.net/man/1/qemu-kvm
• http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?creating-loopback-s3-linux.html
• http://aws.amazon.com/developertools/351?_encoding=UTF8&jiveRedirect=1 The tools download page.
• http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?creating-an-ami.html
• http://docs.amazonwebservices.com/AWSEC2/2011-05-15/CommandLineReference/
• http://www.linuxplanet.com/linuxplanet/tutorials/6546/1
• http://alestic.com/2010/01/ec2-ebs-boot-ubuntu
• http://cssoss.wordpress.com/2010/05/10/eucalyptus-beginner%E2%80%99s-guide-%E2%80%93-uec-edition-chapter-4-%E2%80%93-image%C2%A0management/
• http://rwmj.wordpress.com/2011/05/19/rhel-6-1-is-out-new-virtualization-guide-covers-libguestfs/
• https://help.ubuntu.com/community/UEC/CDInstall
• https://help.ubuntu.com/community/UEC/Images
• http://cloud.ubuntu.com/docs/uec-book/
• http://cloud.ubuntu.com/docs/
• http://home.comcast.net/~fbui/bandwidth.html
• http://www.osc.edu/~troy/lperfex/
• [Home page]

Preping the environment

Installing the required tools

1. yum install qemu-kvm
2. yum install qemu-img

Getting the credentials

Eucalyptus credentials

See: https://help.ubuntu.com/community/UEC/CDInstall#STEP%205:%20Obtain%20Credentials

Creating an image

1. Get the OS
2. create loop back image
3. configure image.
4. bundle image.
5. Install the bundle on UEC.
6. Test the Ubuntu i386 bundle on AWS.
7. Test the bundle on AWS

Installing the tools

Installing the EC2 tools

1. get the tools zip file: EC2 AMI Tools
   • The AMI tools uses ruby: yum install ruby
2. cd /opt
3. unzip ec2-ami-tools.zip
4. export JAVA_HOME=/usr
5. export EC2_AMITOOL_HOME=/opt/ec2-ami-tools-1.3-66634
6. export PATH=$PATH:${EC2_AMITOOL_HOME:-EC2_HOME}/bin

For the API tools

1. export EC2_HOME=/opt/ec2-api-tools-1.4.3.0

• Use the PATH set above. export PATH=$PATH:$EC2_HOME/bin

Configuring the credentials

1. Get the credentials file
   1. firefox https://192.168.1.10:8443
   2. login
   3. click: Download Credentials
   4. save to file
2. install the credentials
   1. unzip -d ~/.euca ~/Downloads/euca2-*-x509.zip
3. update env
   1. echo "[ -r ~/.euca/eucarc ] && . ~/.euca/eucarc" >> ~/.bashrc
4. verify
   1. bash
   2. env | grep EC2
   3. euca-describe-images -a

Installing the admin credentials

1. login and get teh admin credentials
2. unzip -d ~/.euca_admin /home/larkadm/Downloads/euca2-admin-x509.zip
3. . ~/.euca_admin/eucarc

Creating a Fedora image using virt-manager

1. sudo su -
2. yum install @Virtualization
3. chkconfig libvirtd on
4. service libvirtd start
5. virt-manager
6. create guest
   1. select iso install
   2. do not fill entire disc
   3. Use 768 MB mem.
   4. 2GB Storage
   5. select linux and Fedora 15 ad the OS
   6. on last (5 of 5) Expand and select arch i686
7. Install Fedora
   1. Custom storage
      1. / 1500 bytes EXT4
      2. swap (rest)
   2. use 'Minimal'
   3. select 'customize now'
      1. In Base System; select Base.
8. login
9. fix up the configuration
   1. vi /etc/sysconfig/network-scripts/ifcfg-eth0
      • Add BOOTPROTO=dhcp
   2. service network restart
   3. chkconfig network on
   4. vi /etc/selinux/config
      • SELINUX=disabled
10. vi /etc/fstab
    1. • LABEL=uec-rootfs / ext4 defaults 0 0
       • /dev/sda2 none swap sw,comment=cloudconfig 0 0
    2. tune2fs -L uec-rootfs /dev/vda1
    3. reboot
11. Create user: ec2-user
    1. useradd -d /home/ec2-user -c "Default user." -m ec2-user
12. vi /etc/rc.local
13. rm /etc/udev/rules.d/70-persistent-net.rules
14. scp the initramfs and the vmlinuz to the host.
15. halt
    • Halt the guest.
    • The image is in: /var/lib/libvirt/images
16. yum install euca2ools
17. . ~/.euca_admin/eucarc
    • Change to the admin account.
18. cd /tmp
19. Bundle the kernel
    1. euca-bundle-image -i vmlinuz-2.6.38.6-26.rc1.fc15.i686.PAE --kernel true
    2. euca-upload-bundle -b mybucket -m /tmp/vmlinuz-2.6.38.6-26.rc1.fc15.i686.PAE.manifest.xml
    3. euca-register mybucket/vmlinuz-2.6.38.6-26.rc1.fc15.i686.PAE.manifest.xml
       • Save the IMAGE eki number.
20. Bundle the ramdisk image
    1. euca-bundle-image -i initramfs-2.6.38.6-26.rc1.fc15.i686.PAE.img --ramdisk true
    2. euca-upload-bundle -b mybucket -m /tmp/initramfs-2.6.38.6-26.rc1.fc15.i686.PAE.img.manifest.xml
    3. euca-register mybucket/initramfs-2.6.38.6-26.rc1.fc15.i686.PAE.img.manifest.xml
       • Save the IMAGE eri- number.
21. Bundle the image
    1. euca-bundle-image -i larc1.img --kernel eki-44FF1575 --ramdisk eri-DD621799 --arch i386
    2. euca-upload-bundle -b i386 -m /tmp/larc1.img.manifest.xml
    3. euca-register i386/larc1.img.manifest.xml
22. testing
    1. euca-run-instances emi-090A0C1A -k helloworld -t m1.large
    2. euca-describe-instances
       • wait for the instance to come up in running state.
    3. euca-get-console-output i-4CF107FF

Creating a Fedora image using qemu

1. sudo su -
   • if the kvm is run as an ordinary user then there seems to be some timer hw issue in the guest.
2. qemu-img create -f qcow2 diskimage.img 5G
3. qemu-kvm -m 768 -cdrom Fedora-15-i386-DVD.iso -drive file=diskimage.img,if=scsi,index=0 -boot d -net nic -net user
4. go through the RHEL installation
   • select keyboard, language etc.
5. qemu-kvm -m 512 -drive file=diskimage.img,if=ide,index=0 -net nic -net user
6. login
   1. fix up the configuration
   2. vi /etc/sysconfig/networ-scripts/ifcfg-eth0
      • Add BOOTPROTO=dhcp
   3. service network restart
   4. chkconfig network on
   5. vi /etc/selinux/config
      • SELINUX=disabled
   6. reboot
7. ::: install curl

Creating a Fedora image - old school

1. dd if=/dev/zero of=fedora.fs bs=1M count=2048
2. mke2fs -F -j fedora.fs
3. mkdir /mnt/fedora
4. mount -o loop fedora.fs /mnt/fedora
5. mkdir /mnt/fedora/dev
6. /sbin/MAKEDEV -d /mnt/fedora/dev -x console
7. /sbin/MAKEDEV -d /mnt/fedora/dev -x null
8. /sbin/MAKEDEV -d /mnt/fedora/dev -x zero
9. mkdir /mnt/fedora/etc
10. vi /mnt/fedora/etc/fstab
    • See fstab content below.
11. cat /etc/yum.conf /etc/yum.repos.d/fedora.repo >> /mnt/fedora/etc/yum.conf
12. vi /mnt/fedora/etc/yum.conf
    • See yum.conf content below.
13. mkdir /mnt/fedora/proc
14. mount -t proc none /mnt/fedora/proc
15. yum -c /mnt/fedora/etc/yum.conf --installroot=/mnt/fedora -y groupinstall Base
16. vi /mnt/fedora/etc/sysconfig/network-scripts/ifcfg-eth0
    • See blow
17. echo "NETWORKING=yes" > /mnt/fedora/etc/sysconfig/network
18. Update /mnt/fedora/etc/fstab
    • See below
19. chroot /mnt/ec2-fs /bin/sh
20. chkconfig --level 345 my-service on
21. exit
22. umount /mnt/fedora/proc/
23. umount -d /mnt/fedora
24. ec2-bundle-image -i /disk2/fedora.fs -k ~/.euca/mykey.priv -c ~/.euca/euca2-ME-CODE-cert.pem -u 123456789012 -r x86_64 -d /disk2/product

fstab

/dev/sda1  /         ext3    defaults        1 1
none       /dev/pts  devpts  gid=5,mode=620  0 0
none       /dev/shm  tmpfs   defaults        0 0
none       /proc     proc    defaults        0 0
none       /sys      sysfs   defaults        0 0

# This is for c1.small and m1.medium
#  For others please see:
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?instance-storage-concepts.html
/dev/sda2  /mnt      ext3    defaults        0 0
/dev/sda3  swap      swap    defaults        0 0

yum.conf

[main]
cachedir=/mnt/fedora/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
plugins=1
installonly_limit=3
color=never
exclude=*-debuginfo
gpgcheck=0
reposdir=/dev/null

#  This is the default, if you make this bigger yum won't see if the metadata # is newer on the remote and so you'll "gain" the bandwidth of not having to # download the new metadata and "pay" for it by yum not having correct # information.
#  It is esp. important, to have correct metadata, for distributions like # Fedora which don't keep old packages around. If you don't like this checking # interupting your command line usage, it's much better to have something # manually check the metadata once an hour (yum-updatesd will do this).
# metadata_expire=90m

# PUT YOUR REPOS HERE OR IN separate files named file.repo # in /etc/yum.repos.d

[fedora]
name=Fedora $releasever - $basearch
failovermethod=priority
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-14&arch=$basearch
enabled=1
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-debuginfo]
name=Fedora $releasever - $basearch - Debug failovermethod=priority #baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/debug/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-source]
name=Fedora $releasever - Source
failovermethod=priority
[main]
cachedir=/mnt/fedora/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
plugins=1
installonly_limit=3
color=never
exclude=*-debuginfo
gpgcheck=0
reposdir=/dev/null

#  This is the default, if you make this bigger yum won't see if the metadata # is newer on the remote and so you'll "gain" the bandwidth of not having to # download the new metadata and "pay" for it by yum not having correct # information.
#  It is esp. important, to have correct metadata, for distributions like # Fedora which don't keep old packages around. If you don't like this checking # interupting your command line usage, it's much better to have something # manually check the metadata once an hour (yum-updatesd will do this).
# metadata_expire=90m

# PUT YOUR REPOS HERE OR IN separate files named file.repo # in /etc/yum.repos.d

[fedora]
name=Fedora $releasever - $basearch
failovermethod=priority
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-14&arch=$basearch
enabled=1
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-debuginfo]
name=Fedora $releasever - $basearch - Debug failovermethod=priority #baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/debug/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-source]
name=Fedora $releasever - Source
failovermethod=priority
#baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/source/SRPMS/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-source-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch
#baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/source/SRPMS/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-source-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

ifcfg-eth0

DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet
USERCTL=yes
PEERDNS=yes
IPV6INIT=no

Creating an Ubuntu image

• | Building an Ubuntu EMI: Ground Up

1. apt-get install python-vm-builder
2. vi image_def.txt

root 1000
/mnt/ephemeral 2000 /dev/sda2
swap 100 /dev/sda3
 

1. vmbuilder xen ubuntu --part ./image_def.txt
   • clear; vmbuilder qemu ubuntu -d /disk2/tmp/tut --verbose --part ./image_def.txt
2. mkdir /mnt/ubuntu
3. mount ubuntu-xen /mnt/ubuntu -o loop
4. chroot /mnt/ubuntu/ apt-get update
5. chroot /mnt/ubuntu/ apt-get install openssh-server
6. chroot /mnt/ubuntu/ passwd -d root
7. chroot /mnt/ubuntu vi /etc/rc.local
   • Add the code before the “exit 0“

depmod -a
modprobe acpiphp
 
# simple attempt to get the user ssh key using the meta-data service
mkdir -p /root/.ssh
echo >> /root/.ssh/authorized_keys
curl -m 10 -s http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key | grep 'ssh-rsa' >> /root/.ssh/authorized_keys
echo "AUTHORIZED_KEYS:"
echo "************************"
cat /root/.ssh/authorized_keys
echo "************************"
 

1. chroot /mnt/ubuntu apt-get install curl
2. cp 2.6.28-11-generic /chroot/lib/modules -R
3. euca-bundle-image -i ubuntu-xen/root.img --kernel eki-CD7D185A --ramdisk eri-18301945 --prefix vmbuilder-test7
4. euca-upload-bundle -b imagestore-vmbuildertest6 -m /tmp/vmbuilder-test7.manifest.xml
5. euca-register imagestore-vmbuildertest6/vmbuilder-test7.manifest.xml
6. umount -l /mnt/ubuntu

Vanilla image creation

• | Creating an Eucalyptus image
• Make_QEMU_image_with_kickstart

Seems like EKI and ERI are something that is provide from outside the image. So it boot of of a kernel that is provided by the cloud vendor.

1. yum install qemu-img
2. qemu-img create -f qcow2 fedora.img 5G
3. qemu-kvm -m 256 -cdrom ../isos/fedora14.iso -drive file=fedora.img,if=scsi,index=0 -boot d -net nic -net user
   • Why use vnc?: -nographic -vnc :0

creating a RHEL server image on ubuntu

1. sudo su -
   • if the kvm is run as an ordinary user then there seems to be some timer hw issue in the guest.
2. qemu-img create -f qcow2 diskimage.img 5G
3. kvm -m 512 -cdrom rhel-server-6.1-i386-dvd.iso -drive file=diskimage.img,if=scsi,index=0 -boot d -net nic -net user
4. go through the RHEL installation
   1. select keyboard, language etc.
5. ::: install curl

• Eucalyptus beginners guide UEC at http://cssoss.wordpress.com/
• RHEL 6.1 AMI: http://aws.amazon.com/amis/0471856266979567

KVM Issues

qemu network issues

Unable to connect to host os from within the guest

ping hosts does not get a response.

qemu-kvm issues

Boot failed: could not read the boot disk

• if=ide

kvm -m 768 -drive file=diskimage.img,if=scsi,index=0 -net nic -net user

Running a the rescue op from the DVD fdisk provides:

• fdisk -l /dev/sda
  • Partition 1 has different physical/logical beginnigs (non-linux?):
    • phys=(0, 32, 33) logical=(99, 115, 10)
  • Partition 1 does not end on cylinder boundary.

Show the same errors for partition 2, which is the LVM.

qemu-kvm: -net use: Parameter 'type' expects a network client type

• it should be -net user

qemu-kvm -m 768 -cdrom Fedora-15-i386-DVD.iso -drive file=diskimage.img,if=scsi,index=0 -boot d -net nic -net use

Guest issues

Bringing up interface eth0: Device eth0 has different MAC address than expected, ignoring.

remove HWADDR from /etc/sysconfig/network-scripts/ifcfg-eth0

Yum issues

Cannot retrieve repository metadata (repomd.xml) for repository

yum -c /mnt/fedora/etc/yum.conf --installroot=/mnt/fedora -y groupinstall Base 

Loaded plugins: langpacks, presto, refresh-packagekit Error: Cannot retrieve repository metadata (repomd.xml) for repository: fedora. Please verify its path and try again

Image management

1. euca-add-keypair helloworld > ~/.euca/helloworld.priv
2. euca-describe-images
3. euca-run-instances emi-1E9911E8 -k helloworld -t c1.medium
4. euca-describe-instances
5. euca-get-console-output
6. ssh -i ~/.euca/helloworld.priv ec2-user@172.16.1.100
7. euca-terminate-instances

remove image

1. euca-deregister emi-09AF0C32
2. euca-delete-bundle -b i386 /tmp/larc3.img.manifest.xml

Cloud issues

Permission denied (publickey).

• A: user name was wrong.
  • Correct: ssh -i ~/.euca/helloworld.priv ubuntu@172.16.1.100

• ssh -i ~/.euca/helloworld.priv user@172.16.1.100

 ssh -i /home/larkadm/.euca/helloworld.priv user@172.16.1.100 -v
OpenSSH_5.5p1 Debian-4ubuntu6, OpenSSL 0.9.8o 01 Jun 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 172.16.1.100 [172.16.1.100] port 22.
debug1: Connection established.
debug1: identity file /home/larkadm/.euca/helloworld.priv type -1
debug1: identity file /home/larkadm/.euca/helloworld.priv-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.8p1 Debian-1ubuntu3
debug1: match: OpenSSH_5.8p1 Debian-1ubuntu3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.5p1 Debian-4ubuntu6
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '172.16.1.100' is known and matches the RSA host key.
debug1: Found key in /home/larkadm/.ssh/known_hosts:8
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/larkadm/.euca/helloworld.priv
debug1: read PEM private key done: type RSA
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).