StatelessLinux/iSCSIRoot

StatelessLinux - iSCSI Root

See Stateless Linux HOWTO

Another option which is quite similar to NFS root is to export the ext3 image file as an iSCSI LUN. Again, clients boot using PXE but the initrd downloaded sets up the iSCSI device and mounts the root filesystem from it.

Creating The initrd

Firstly, you need to install iscsi-initiator-utils and set up the iSCSI device. Setting up the device involves discovering and logging into the target.

$> yum install iscsi-initiator-utils
$> iscsiadm -m discovery -t st -p 172.31.0.7
172.31.0.7:3260,1 iqn.1994-06.com.redhat.devel:markmc.test1
$> iscsiadm -m node -p 172.31.0.7 -T iqn.1994-06.com.redhat.devel:markmc.test1 -l

Next, mount the image and set up the chroot which you'll use to create the initrd:

$> mount -o ro /dev/sda tmpmount/
$> mount -t tmpfs none tmpmount/tmp
$> mount -t tmpfs none tmpmount/dev
$> mount -t tmpfs none tmpmount/var/lock
$> mount -t tmpfs none tmpmount/var/lib/iscsi
$> mount -t proc none tmpmount/proc
$> mount -t sysfs none tmpmount/sys

Discover the target in the chroot:

$> chroot tmpmount/ iscsiadm -m discovery -t st -p 172.31.0.7

Now run mkinitrd:

$> chroot tmpmount/ mkinitrd -with crc32c --with e1000 --net-dev eth0 \
--rootdev /dev/sda --rootfs ext3 -f \
/tmp/initrd.img 2.6.17-1.2693.fc6
$> mv tmpmount/tmp/initrd.img .

Clean up after yourself:

$> for dir in sys proc var/lock var/lib/iscsi dev tmp; do umount tmpmount/$dir; done
$> umount tmpmount/
$> iscsiadm -m node -p 172.31.0.7 -T iqn.1994-06.com.redhat.devel:markmc.test1 -u

Notes:

• We pass --with crc32c to mkinitrd because of bug #208607

Setting Up A Testing Target

Most people will only use iSCSI root with real iSCSI devices, but for testing purposes it can be quite useful to set up an iSCSI target on your image server.

One option is to use NetBSD project's user space iSCSI initiator from from netbsd-iscsi package, the other one is iSCSI Enterprise Target that is available in Livna repository as iscsitarget. Advantage of NetBSD iSCSI initiator is that it doesn't need a kernel module, on the other hand it doesn't give you possibility to arbitrarily name the target (has to be called target0, target1, ...), and IET is more likely to perform better.

If you don't want the Livna package, you can build IET it from source code:

First, build a Fedora kernel and then build IET, setting KSRC to point to the location of your kernel build:

$> svn co http://svn.berlios.de/svnroot/repos/iscsitarget/trunk
$> make KSRC=/usr/src/redhat/BUILD/kernel-2.6.17/linux-2.6.17.x86_6

Load the iscsi_trgt kernel module:

$> insmod ./kernel/iscsi_trgt.ko

Create an ietd.conf config file:

$> cat > ./test-ietd.conf << EOF
Target iqn.1994-06.com.redhat.devel:markmc.test1
Lun 0 Path=/stateless/terminal/terminal.img,Type=fileio
EOF

Start ietd:

$> ./usr/ietd -f -d 8 -c ./test-ietd.conf

Open TCP port 3260 in your firewall:

$> cat >> /etc/sysconfig/system-config-securitylevel << EOF
--port=3260:tcp
EOF
$> lokkit --quiet

Notes:

• The developer of IET is also helping to develop Linux SCSI target framework (stgt) which looks like it might lead to an iSCSI target implementation with an upstream kernel component. There's still some debate about how the kernel component should actually work and the prototype iSCSI implementation doesn't work.

• mkinitrd-5.1.17 has a bug (bug 208383 ) where if you create the initrd on the same machine as the iSCSI target, the initrd will try to bring up the loopback network device on the client.

• mkinitrd-5.1.17 also has a problem (bug 209110 ) whereby if the iSCSI target device is available as anything other than /dev/sda on the machine where the initrd is created, then the client won't boot. You can workaround this by booting the client with root=/dev/sda on the kernel command line.

• Don't be surprised if you see an iSCSI related kernel oops when booting the client. That's currently the case when booting in a Xen guest with kernel-xen-2.6.17-1.2693.fc6