How To Run the ARMv8 FAST or Foundation Model with an NFS Root
Here's all the ingredients you're going to need:
- A root file system to export via NFS (and an NFS server for it, of course)
- A network bridge configuration that allows for NFS traffic, DHCP traffic, and traffic to the big bad Internet.
- A running version of the ARM FAST or Foundation Models (FM).
- An "application image" for the FM to run.
What follows is the setup I (Al Stone) am using that appears to work reasonably well. Caveat: I have not set up network routing yet so that I can get to the model; the model can get out while running, but you can't get in so it's likely just a routing problem.
NB: all scripts mentioned here are in http://fedorapeople.org/groups/armv8 for your edification and amusement
NFS Root File System
You'll need to make sure nfs-utils is installed and that your firewall (if any) will allow NFS mounts to actually happen.
NB: when shutting down the FM you must halt it (with the actual halt command). If you do not, you end up with NFS ghost mounts. You will be able to run the FM again, though, and it will be able to reuse the mount point.
Before you can provide the NFS mount point, you need to create the root file system ("rootfs") to export. While there are a couple available, the one we're most interested in (for obvious reasons) is the stage2 bootstrap rootfs we're working on for aarch64 -- a version of Fedora17, more or less (see [[Architectures/ARM/AArch64/Stage2Notes| ARMv8 Bootstrap Stage2 Notes). You can retrieve a copy of the current state of the rootfs from git and stash it somewhere handy:
$ mkdir ~/armv8-model $ cd ~/armv8-model $ git clone http://fedorapeople.org/groups/armv8/rootfs.git nfs-root
You can change the location, but you will need to adapt the scripts and commands below to account for your changes. We now have a base file system to export. Add a line to /etc/exports that looks like this (in my case, at least):
/home/ahs3/armv8-model/nfs-root *(rw,no_root_squash,no_all_squash,async)
Start up NFS:
$ sudo systemctl start nfs.service $ sudo exportfs -rv $ sudo showmount -a # should at least tell you there are no mounts
And that's that. Test it out by mounting it either locally or from another system; if it does not work, check your firewall, and make sure NFS is actually running.
Networking
This is where it all gets messy. Basically, you're attempting to do the same thing libvirt and/or LXC and/or KVM are doing when they attempt to run a virtual machine on your desktop.
I've created some scripts to do this for me (once I figured out what would work in my environment) that of course can be re-used, but you may have to play with them to use them in your particular corner of the Internet:
- setup-bridge.sh => create all the pieces temporarily
- shutdown-bridge.sh => destroy all the pieces
The most straightforward thing I found to do was statically configure all of the IP addresses; DHCP just made it difficult to hunt down problems. Further, FM has it's own internal DHCP server that can just confuse things a bit more. Patches are welcome.
The setup script:
- creates a bridge called armbr0 and assigns it an IP and mask useful on your local network, then
- creates a TUN device called ARM${USER} and connects it to the bridge, then
- starts up dnsmasq on the armbr0 to provide DNS, DHCP, and aliasing services (though we only end up using the aliasing), and finally
- turns on IP forwarding and sets iptables to NAT everything attached behind the bridge.
The tun device name is essential to running the FM -- you'll need it to tell the FM what network device to use.
NB: dnsmasq and NAT'ing may be optional; that's TBD yet. It works right now so I haven't experimented with simplifying it.
In my case, I put the bridge and it's devices on the 172.17.100.0/24 subnet (which makes sense in my network). You can't use the 10.net since the RH VPN will take over all routing to the 10.net. The bridge gets an address of 172.17.100.1 with a mask of 255.255.255.0.
The setup-bridge.sh script also uses a file dnsmasq.conf. This file is pretty important: it tells dnsmasq how to map the NFS root requests it receives to an actual location. You'll need the IP address of the NFS server (there is no DNS lookup available at boot) and the mount point. In my case, the file looks like this:
dhcp-option=17,"172.17.2.15:/home/ahs3/armv8-model/nfs-root"
The quotes are critical, and it must be option 17.
To get the bridging set up (you will need sudo permissions):
$ ./setup-bridge.sh
Once you're finished, you can tear it all down with:
$ ./shutdown-bridge.sh
Running the FAST Model
There's three parts to this:
- Getting a license file -- an exercise left to the reader.
- Installing the FM -- untar the tarball from ARM (you'll need to figure out how to get the tarball) and look for 'setup.bin' and then run it. If you're on an x86-64 system, you will need to install a bunch of 32-bit libraries to get it to work. In my case, I installed the FM in ~/armv8-model.
- Running the license manager. The license file tells the daemon 'lmgrd' where the license server is; getting this set up is also an exercise left to the reader.
While I could document this in better detail, I'm not going to at present since this is tightly controlled software (ARM is understandably careful about distribution). If you really need the FAST model, please contact ARM for more details. In general, the Foundation model (see next section) should be sufficient.
For the sake of argument, let's assume you can run the FM (or the Foundation model) for now.
Running the Foundation Model
This route is much simpler; there are far fewer controls on the software by ARM. To retrieve a copy (you will need to provide a login to the site), please go to: http://www.arm.com/fvp
Towards the bottom of that page is a link to follow to register and then download a copy of the Foundation model (also FM). It is free to use -- assuming you're willing to register with the site -- but I have not examined the license to determine if it is redistributable. You'll end up downloading a file with a name such as FM000-KT-00035-r0p8-44rel23.tgz. For simplicity's sake, I place this in the same location as my rootfs and then unpack it:
$ cd ~/armv8-model $ tar xzf FM000-KT-00035-r0p8-44rel23.tgz
The binary that is the FM will then be in ~/armv8-model/Foundation_v8pkg/Foundation_v8. An alias is sometimes useful:
$ alias fv8=~/armv8-model/Foundation_v8pkg/Foundation_v8
If you then type 'fv8 --help' and get all sorts of stuff, you can run the FM.
Getting and Booting an Application Image
The ARM models use something called an "axf" file that contains all the components needed to boot an instance of the platform simulator -- kernels, initrds and so on. These axf files unfortunately need licensed code from ARM to produce. If everything above is working, and you're using exactly the same networking setup as in the scripts attached, and you have set up the bridging, you should be able to retrieve the axf file that has already been prepared and have an xterm pop up that is the console for the ARMv8 FAST model being run by using these commands:
$ cd ~/armv8-model $ scp username@fedorapeople.org:/project/armv8/linux-system-nfs-rh.axf . $ scp username@fedorapeople.org:/project/armv8/run-nfs-rootfs-rh.sh . $ ./run-nfs-rootfs-rh.sh
If you have the FAST model and can build axf files, and you want to reproduce the .axf file exactly, the cmdline.nfs used looks like the following:
root=/dev/nfs nfsroot=172.17.100.1:/home/ahs3/armv8-model/nfs-root-arm,tcp,v3 rw ip=172.17.100.2:172.17.100.1:172.17.100.1:255.255.255.0:aarch64:eth0:none console=ttyAMA0 mem=2G earlyprintk debug maxcpus=2Addresses can of course be changed, but you will also need to make sure the bridge setup and run-nfs-root-rh.sh all reflect any changes you have made.
Once the script runs, you should see a message go by in the xterm indicating that an NFS root is being used. After the the "Freeing init memory" message, you should see a prompt asking you to hit enter to start a session. So do it. And now you're running the FM using the f17 rootfs with an NFS root file system. This is a very limited Linux system (it's basically BusyBox); the good news is that since the rootfs is in git, you can now add to it as more of stage2 gets built simply by using 'git pull --all'.
To use the Foundation model instead, you will have still had to setup the bridge, and the NFS exports. And, you'll need the same .axf file as above for the FAST model. The difference is in how you execute the model, which changes to this:
$ cd ~/armv8-model
$ fv8 --image linux-system-nfs-rh.axf --network=bridged --network-bridge=ARM${USER}0
(Obviously, replace ${USER} with your user name). Again, you'll see a console pop up, it'll prompt you to hit enter to start a session, so you just do it and off you go.