Anaconda/Stage2DevelopmentGuide

What's needed for working on stage2

There are a couple of things you need to work on stage2 that make development go much more quickly.

• anaconda source tree: Obviously, you need the correct version of the anaconda source code. Usually you will want the latest checkout from the appropriate branch.

• Writeable NFS volume: You'll need some NFS space that you can write to. At the least, you will need to be able to create an RHupdates directory and add files to it. You may sometimes also want to modify the packages in the tree (like for testing failure cases). You'll want to make sure this NFS volume is the same version as what you're trying to debug. So if you are working on a RHEL5 bug, make sure you have a RHEL5 tree. If it's a Fedora development bug that only happens on a certain day's tree, make sure you have that day's tree. These versions don't always have to match up exactly.

• Web space: Having some publicly available web space is helpful for posting updates.img and kickstart files. These are helpful both for your development and for distributing to other people for testing.

• CD/DVD media: For testing media installs, make sure you have burned the latest available media. These are available for all releases, beta releases, and some nightly trees around release time. However, they are not available for all nightly trees.

• HTTP/FTP server: For testing there remote install methods, you'll need a server running that has the correct trees available. These methods are not tested all that often, so this isn't typically all that important. Close to a release, you will want to have access to this however.

Coding Standard

When in doubt about Python coding standard we use just go with the style in the file you are changing. If creating new file or still in doubt, consult PEP8.

Testing Changes

By far, the easiest way to test your changes is to use the NFS installation method. To test your modified files, you then create an RHupdates directory in the top level of the installation tree. That's the level that contains the license file, release notes, and GPG keys. Then put your modified files into that directory and reboot your test machine. anaconda will automatically detect the presence of the RHupdates directory.

Files in RHupdates or an updates.img are used instead of the same files from the stage2 image. If your modifications include completely new files or different versions of shared libraries, you can include those in RHupdates. Due to the way python modules get imported, if you need to include an updated file from a python module you will need to include all the files from that module. However, anaconda has a shortcut for certain modules that we control and update frequently that allows you to just put the changed files in your update. These modules are listed in setupPythonUpdates in the anaconda file.

Some bugs only show up on specific install methods. When that install method isn't NFS, you can't use the RHupdates directory. You'll need to make an updates.img containing all your modifications. An updates.img is an ext3 filesystem image that can contain anything RHupdates can.

Typically, anaconda cannot automatically detect an updates.img. You'll need to provide the updates command line parameter if the image is on a floppy or USB key drive, or updates= if you have put your image on a web server. Updates via HTTP or FTP are often the easiest way to go as you can work up a fix, test it, and then point extra testers at the location.

Note that updates= only works on RHEL5 and later, and all recent Fedora releases. It is not supported in RHEL4 so you will have to use a floppy or USB key drive.

See AnacondaUpdates for more explanation on updates.img.

This script generates an updates.img containing all the files listed on the command line and uploads it to a publicly accessible website for testing:

#!/bin/bash
#

MOUNTPOINT=/misc

if [[ $# < 2 ]  ; then
echo "usage: $0 <updates.img filename> <file1> ... <filen>"
exit
fi

if [ -d $MOUNTPOINT/lost+found ] ; then
echo "/misc is already mounted, exiting"
exit 1
fi

UPDATES_IMG=$1
shift

if [ -f /tmp/$UPDATES_IMG ] ; then
rm -f /tmp/$UPDATES_IMG && exit 1
fi

dd if=/dev/zero of=/tmp/$UPDATES_IMG bs=1M count=1
yes | /sbin/mkfs.ext3 /tmp/$UPDATES_IMG
mount -o loop /tmp/$UPDATES_IMG $MOUNTPOINT
cp -a $* $MOUNTPOINT
umount /tmp/$UPDATES_IMG

scp /tmp/$UPDATES_IMG clumens@people:public_html/$UPDATES_IMG
rm -f /tmp/$UPDATES_IMG

Alternative version, which checks adds modified files from your git repository or files modified in a specific commit. Does not need root privileges, since it uses CPIO format instead of a filesystem image.

#!/bin/bash
# Usage: updates.sh [<commit>]

set -e

COMMIT=$1
if [ -n "$COMMIT" ]
then
        FILES=$(git show $COMMIT |sed -n 's,^+++ b/\(.*\),\1,p')
else
        FILES=$(git status |awk '/modified:/ {print $3}')
fi

if [ -z "$FILES" ]
then
        echo "No files" >&2
        exit 1
fi

(ls $FILES |cpio -c -o) |gzip -c9 >updates.img
scp updates.img fedorapeople.org:public_html
rm updates.img

Debugging

One of the primary ways to debug anaconda is by using pdb, the interactive Python debugger. This guide assumes you already know how to use pdb. If not, see the pdb reference guide.

Python Debugger

Python's interactive debugger is not nearly as fancy or useful as gdb, but it has enough features for us to figure out what's going on. The most common way you will interact with pdb is by pressing the Debug button on anaconda's exception dialog box. Pressing this immediately drops you into pdb on tty1. From here, you can inspect values and check out the state of the filesystems. However, you cannot return to anaconda. Once in this top-level exception handler, continuing will cause anaconda to exit.

You can also enter pdb at any point by adding a function call exactly where you need it and putting that updated file in RHupdates. Let's say you were debugging a problem where anaconda was not writing out the anaconda-ks.cfg file towards the end of installation. This code is located in packages.py and looks like this:

def writeKSConfiguration(anaconda):
    log.info("Writing autokickstart file")
    if not flags.test:
        fn = anaconda.rootPath + "/root/anaconda-ks.cfg"
    else:
        fn = "/tmp/anaconda-ks.cfg"

    anaconda.id.writeKS(fn)

You could drop into the debugger at the beginning of this function by changing it to the following:

def writeKSConfiguration(anaconda):
    import pdb
    pdb.set_trace()

    log.info("Writing autokickstart file")
    if not flags.test:
        fn = anaconda.rootPath + "/root/anaconda-ks.cfg"
    else:
        fn = "/tmp/anaconda-ks.cfg"

    anaconda.id.writeKS(fn)

The only changes here are the new two lines at the beginning of the function. Once anaconda reaches this point, it will drop to a pdb prompt on tty1. If you're watching the graphical interface, you will notice that it has stopped updating when it reaches this point. Then, you'll need to switch over to tty1 and do whatever debugging actions you want. Unlike being in pdb from the exception dialog, here if you continue anaconda will keep going as expected.

This is one of our primary debugging techniques.

There's a third way to get into the Python debugger. By adding the debug command line parameter, you will get a Debug button on every screen in the interface and on certain dialogs. Pressing this button automatically drops you into pdb. From here, you can debug and then continue if needed.

Logging

anaconda includes a logging framework that supports multiple severity levels, multiple logging destinations, and logging to a remote server. The stage2 supports all these features, while the loader only supports multiple severity levels. By default, anaconda logs to tty3 and to the /tmp/anaconda.log file. The severity level controls what messages will be written to tty3. All log messages regardless of severity level will be written to /tmp/anaconda.log. Severity is controlled by the loglevel= command line parameter. Valid settings are debug, info (the default setting), warning, error, and critical.

Remote logging is more useful for users on headless systems than it is for developers. However there are still times when it's handy. To log stage2 messages to a remote system, use the syslog=host[:port] parameter as discussed in Anaconda/Options . Remote logging is subject to the severity level system as well.

Often when debugging a problem, you won't get all the output you need from anaconda. You can easily add some extra logging statements to the code where you need it. First, make sure the python file imports the logging module. Most do, but the ones that don't will need a block like the following near the top:

import logging
log = logging.getLogger("anaconda")

Then add your logging statements throughout. Each logging statement has to specify its severity level. For debugging, it's easiest to just use error and not worry with the loglevel parameter.

log.error("error unmounting filesystem in systemUnmounted")

Causing exceptions

Rarely, it can be useful to cause an exception. The most common time when you may want to do this is when testing out the exception handlers. The easiest way to do this is to just add in a

raise SystemError

right in the code at the point where you'd like an exception.

Shell

There is a terminal usually located in tty2 that is very useful when you want to see something in the file system during install. It has some basic commands, not much (Remember that nothing has been installed yet). A lot of the python code base is included so you can wget any script and run it if you need to. Its a very good way to poke around and see whats happening.

Remote Access

In F13 and later, the 'sshd' parameter on the kernel command line may be used to start a (passwordless) ssh server on the machine once stage2 is running. This is useful for cases where X appears to lock up or where VT switching doesn't work for some other reason. Earlier releases can enable sshd in the kickstart file.

Note that you'll have to be able to predict or deduce the IP address that the installing machine will get. If the machine under test is a KVM or Xen guest, you can look at the ARP cache on the host with 'arp -n' to find it, usually 192.168.122.*. If it's a physical machine, watch the logs of the appropriate DHCP server, or else explicitly configure networking in stage1.

Other Log Files

There are a couple other log files that may be useful.

• /tmp/anaconda.log: This is the primary log file for anaconda, containing all log messages regardless of severity level.

• /tmp/anacdump.log: When an exception occurs, anaconda will automatically write this file out. It contains the python traceback, the anaconda.log, kernel messages, and a dump of all python variables. This is a very large file. When people file bug reports, they will usually attach this file to the bug report. If they don't ask for it as this is the main debugging information we need.

• /tmp/X.log: This is the log file from when X starts up. If there are problems starting X, this file will help us figure out what's going on. These are usually from bugs in X itself but the X team will ask for this information so it's good to get it up front from the user.

---