QA/Automated Installer Testing

These are the current plans for automated testing of the installer. The specific goal is this: perform daily, basic, fully-automatic installations of Rawhide, and report the results to a public web page.

Hardware

• One server machine with enough disk space for a local mirror of rawhide
  • Roughly 15G per arch. Ouch.
• One test machine per arch - i386, x86_64, and ppc.
  • ppc is considered almost optional.
  • KVM/QEMU/Xen could be used on the server machine to test in virtual guests.
  • Must be PXE-capable
• Some form of power control
  • This could be non-automated - "if the machine is dead, hit the power button".

Software

• cobbler and koan
  • See http://fedorahosted.org/cobbler/ for more details
  • Note that snake is still being considered but cobbler/koan seem to be much better.
• Rawhide sync script
  • http://jkeating.fedorapeople.org/sync-rawhide.py is an example script
  • Or you could just let cobbler handle it.
• logging client/server
  • jlaska maintains a cobbler git branch that adds automated remote logging of installs (and PPC support!): git://fedorapeople.org/~jlaska/cobbler.git.
  • Enhanced logging patchset has been submitted to cobbler upstream for inclusion in 1.5: https://fedorahosted.org/pipermail/cobbler/2008-December/001942.html

Overall design

1. Daily rawhide compose completes. We get signal.
2. Rawhide gets synced up.
3. Each client machine is instructed to run koan to install the new rawhide.
   • This might happen via func or some other mechanism.
4. A %pre script in the kickstart runs the logging client
   • Watch the following log files:
     • /tmp: anaconda.log, syslog, ks-script.log, anacdump.txt, modprobe.conf
     • /mnt/sysimage/root: install.log, upgrade.log
   • This is all handled by the 'anamon' code in jlaska's cobbler git tree.
5. If something fails during install
   • anacdump.txt appears, or no log data for 30min
   1. Use cobbler to change the PXE target to reinstall stable OS
   2. Use power control to restart the machine
   3. Close log task with status FAIL
6. Otherwise: %post script in kickstart creates a service to run on firstboot
   • Send system startup logs (to be checked for errors)
   • Send signal to logging server to close task with status OK
7. Logging server checks over collected logs for errors
   • avc errors in /var/log/messages, FAILED in /var/log/boot.log, etc.
   • Change status to WARN if found.