AutoQA Self Testing

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Introduction

Self tests were introduced into AutoQA with the addition of some unit tests in the 0.5.0 release. The work on the self testing facilities for AutoQA is far from finished and this is just a step in the right direction.

The structure and process for running these tests will evolve over time as we improve the testability of AutoQA and expand on the few tests that are already written.

Tools Used

Testing frameworks for use with AutoQA have already been evaluated and keeping with the conclusions from that evaluation, py.test is being used to run the tests.

Required Packages
Both python-virtualenv and python-pip are required in order to run make test

Since py.test > 2.0 is still not packaged in Fedora at the time of writing, we needed to do some hacking in order to make things work. This involved the use of additional tools:

• python-virtualenv
  
• python-pip
  

Process

Make Test
This process surrounding running AutoQA's self-tests need not be run manually. It can be run with make test

The basic idea is to create a virtualenv with autoqa and py.test for testing to work around the fact that the currently packaged version of py.test is old.

Virtualenv Detection

If the virtualenv is already created, there isn't much sense in creating it again. This is done in a rather simplistic fashion by looking for a directory with the name 'test_env' and if such a directory exists, a properly setup virtualenv is assumed.

If the directory does not exist, a new virtualenv is created and populated with py.test (using pip) and autoqa (via symlink).

Running the Tests

To run the tests, the virtualenv is activated and py.test is invoked. Once the tests have completed, the virtualenv is deactivated and the shell is returned to its previous state.

Gotchas

One potential snag is that this method of creating a virtualenv is uses the same python version that is running on the host system. If your setup has the same directory shared between hosts that have a different version of python, you'll have to delete the test_env directory before switching python versions.

The Future

This current method is a bit of a hack and once a newer version of py.test is packaged for fedora (awaiting review at the time of this writing), the process will change to be much less complicated.

References

<references>