Autotest

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

What is autotest?

Autotest is "a framework for automated testing". It gives us a way to run automated tests and collect results. For more information, see their wiki: http://autotest.kernel.org/

How is it used in Fedora?

We're currently using it as the test harness for AutoQA. It handles the dirty work of getting code onto test machines, running it, and holding all the results.

Currently there is no public autotest instance. The QA team has been working on packaging autotest for use in the Fedora infrastructure.

The AutoQA project is starting out by using it to automate some existing tests (like repoclosure from yum-utils) and some new tests from the Israwhidebroken.com Proposal.

How do I write a test for autotest?

Start by writing some test code in whatever language(s) you feel comfortable with. Don't even worry about autotest or AutoQA until you have a functioning test. Once you do, see Writing AutoQA Tests for more info.

How do I set up my own autotest server?

Docs in progress
See this trac ticket for more details.

Autotest is currently packaged for EPEL, but due to library dependencies, the package is not yet available for Fedora. The following section describes how autotest is installed on a Red Hat Enterprise 5 (or CentOS 5) system using the EPEL packages.

1. Setup the require package repositories
   1. Configure your system to receive RHEL or CentOS package updates through the update mechanism provided by the distribution (e.g. for RHEL, run rhn_register).
   2. Configure the system to receive updates from EPEL, for more information see EPEL/FAQ#howtouse
   3. Create a yum repo for the fedora-infrastructure package repository

      # cat << EOF > /etc/yum.repos.d/fedora-infra.repo

      [fedora-infra]

      name=fedora-infra

      baseurl=http://infrastructure.fedoraproject.org/5/\$basearch

      gpgcheck=0

      EOF

2. Download the latest autotest and the required Django packages:

   <pre>

   # wget http://jkeating.fedorapeople.org/infra/autotest-0.11.0-1.el5.noarch.rpm \

   http://jkeating.fedorapeople.org/infra/Django-0.96-1.20071126svn.ep5.el5.noarch.rpm \

   http://jkeating.fedorapeople.org/infra/autotest-0.11.0-1.el5.noarch.rpm

3. Install the packages using yum to assist with dependency resolution

   # yum --nogpgcheck localinstall autotest-*.rpm Django-0.96*.rpm

4. Start and enable the httpd

   # service httpd restart

   # chkconfig httpd on

5. (optional) - Autotest requires access to a mysql server. You can use an existing server, or setup a new mysql server using the instructions below.
   1. Install and setup mysql server

      # yum install mysql-server

      # chkconfig mysqld on

      # service mysqld start

   2. Setup a password for the root database user. Please use a password other than NEWPASSWORD.

      # mysqladmin -u root password NEWPASSWORD

6. Using the mysql command, login to the root database using the password specified above

   $ mysql -u root -p

7. Create the databases and user accounts needed by autotest using the following SQL commands. Again, please use a password other than NEWPASSWORD.

   create database autotest_web;

   grant all privileges on autotest_web.* TO 'autotest'@'localhost' identified by 'NEWPASSWORD';

   grant SELECT on autotest_web.* TO 'nobody'@'%';

   grant SELECT on autotest_web.* TO 'nobody'@'localhost';

   create database tko;

   grant all privileges on tko.* TO 'autotest'@'localhost' identified by 'NEWPASSWORD';

   grant SELECT on tko.* TO 'nobody'@'%';

   grant SELECT on tko.* TO 'nobody'@'localhost';

   flush privileges;

   exit;

8. Update the autotest configuration files /usr/share/autotest/global_config.ini and /usr/share/autotest/shadow_config.ini, changing the values for password, hostname and notify_email.
9. Run DB migration script to set up DB schemas and populate initial data.

   # python /usr/share/autotest/database/migrate.py --database=AUTOTEST_WEB sync

   # python /usr/share/autotest/database/migrate.py --database=TKO sync

10. Run the Django syncdb operation

    # /usr/share/autotest/frontend/manage.py syncdb --noinput

11. Make sure that current database users have admin privileges

    # mysql -u root -p -e 'update autotest_web.auth_user set is_superuser=1;'

How do I add test clients to a autotest server?

FIXME
This needs more love. Do we want to discuss labels and platforms in this document?

For any test clients created through the administrative interface, the autotest service requires passwordless logins in order to access remote test clients. An RSA key has been created for the autotest user to facilitate passwordless logins. For any test clients you wish to enable passwordless logins on, type:

# su - autotest
# ssh-copy-id -i $HOME/.ssh/id_rsa.pub  root@client.example.com