StatelessLinux/NFSRoot

StatelessLinux - NFS Root

See Stateless Linux HOWTO

NOTE: Much of this is probably out of date now and it needs a complete review/revamp. Note that it also suggests modifying /etc/sysconfig/mkinitrd, which is wrong for a stateless environment.

This is probably the most straightforward way for stateless clients to use a root filesystem image which you have prepared.

Each client PXE boots (i.e. obtains the location of its bootloader via a DHCP request which in turn downloads a kernel and initrd using TFTP) and mounts its root filesystem over NFS.

Create An initrd

In order for the client to mount the NFS exported directory as its root filesystem, it must have an initrd which knows how to do it. The initrd must load the appropriate network card driver, initialise the device using DHCP and mount the root filesystem from the correct NFS server and export.

First, we mount the image and setup /tmp, /proc, /sys etc. for mkinitrd

$> losetup /dev/loop0 ./terminal.img
$> mount -o ro /dev/loop0 nfs-export/
$> mount -t tmpfs none nfs-export/tmp
$> mount -t tmpfs none nfs-export/dev
$> mount -t tmpfs none nfs-export/var/lock
$> mount -t proc none nfs-export/proc
$> mount -t sysfs none nfs-export/sys

Next, in /etc/sysconfig/mkinitrd, configure the network device and driver modules which the clients will use:

$> mount -o remount,rw nfs-export/
$> cat > /etc/sysconfig/mkinitrd << EOF
NET_LIST="eth0"
MODULES="e1000"
EOF
$> mount -o remount,ro nfs-export/

Now run mkinitrd:

$> chroot nfs-export/ mkinitrd --rootdev 172.31.0.7:/stateless/terminal/nfs-export \
--rootfs nfs -f /tmp/initrd.img 2.6.17-1.2510.fc6
$> mv nfs-export/tmp/initrd.img .

Clean up after yourself:

$> for dir in sys proc var/lock dev tmp; do umount nfs-export/$dir; done
$> umount nfs-export/
$> losetup -d /dev/loop0

Notes

• Make sure you are using mkinitrd >= 5.1.19-1
• The --rootdev and --rootfs options are only available in mkinitrd >= 5.1.0-1

Boot A Client

First, you need to make the appropriate kernel and initrd available to pxelinux and create a pxelinux config file pointing at them:

$> cp vmlinuz initrd.img /tftpboot/linux-install
$> cat > /tftpboot/linux-install/pxelinux.cfg/default << EOF
default linux
kernel vmlinuz
append initrd=initrd.img selinux=0
EOF

Next, export the root filesystem over NFS:

$> cat > /etc/exports << EOF
/stateless/terminal/nfs-export 172.31.0.0/24(ro,async,no_root_squash)
EOF
$> exportfs -r

Finally, configure the client's BIOS to boot via PXE and you should see it use DHCP to configure its NIC and obtain the location of pxelinux, download pxelinux using TFTP, download the pxelinux config file, downloaed and boot the kernel and initrd which will in turn mount the root filesystem over NFS.

Notes:

• You may override the NFS server name and mount (i.e. what was passed to --rootdev above) by adding root=myserver:/foo/bar to client's pxelinux configuration

• We boot with selinux=0 since NFS doesn't have support for xattrs which SELinux requires in order to operate correctly.

• You can have a different pxelinux configuration for each client. pxelinux basically looks for a filename based on the client's MAC address, followed by files based on the client's IP address, followed by the default file. See pxelinux.doc in the syslinux package for more details. Here's a little python snippet for generating the MAC address based filename:

def mac_address_to_filename (mac_address):
return "01-" + mac_address.lower ().replace (":", "-")

Boot With Xen

You can also boot a client with NFS root using Xen.

First, though, you need a different kernel and initrd:

• The kernel needs to be a Xen kernel - i.e. from the kernel-xen package

• The initrd needs to contain modules built for that exact kernel

• The network interface will be Xen's virtual interface, so you need the xennet module to be loaded - i.e. set MODULES="xennet" in /etc/sysconfig/mkinitrd before running mkinitrd

Using xm:

$> cat > ./Terminal.cfg << EOF
name = "Terminal"
memory = "512"
vif = [ "" ] 
kernel = "/stateless/terminal/vmlinuz"
ramdisk = "/stateless/terminal/initrd.img"
extra = "selinux=0"
EOF

Using libvirt:

$> cat > ./Terminal.xml << EOF
<domain type='xen'>
<name>Terminal</name>
<os>
<type>linux</type>
<kernel>/redhat/stateless/terminal/vmlinuz</kernel>
<initrd>/redhat/stateless/terminal/initrd.img</initrd>
<cmdline>selinux=0</cmdline>
</os>
<memory>524288</memory>
<vcpu>1</vcpu>
<devices><interface/></devices>
</domain>
EOF
$> virsh create ./Terminal.xml
$> xm console Terminal

Notes:

• The vif configuration in Terminal.cfg above requests that xend generate a random MAC address for the interface. It might make life easier if, after the first run, you specify the generated MAC address using vif = [ "mac=foo"] . That way your DHCP server won't run out of leases.

• Similarily with libvirt, you might want to do <interface><mac address="foo"/></interface>