(early initial structure and some content) |
m (removing draft notice, adding category) |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
= Introduction = | = Introduction = | ||
Self tests were introduced into AutoQA with the addition of some unit tests in the 0.5.0 release. The work on the self testing facilities for AutoQA is far from finished and this is just a step in the right direction. | Self tests were introduced into AutoQA with the addition of some unit tests in the 0.5.0 release. The work on the self testing facilities for AutoQA is far from finished and this is just a step in the right direction. | ||
| Line 6: | Line 5: | ||
= Tools Used = | = Tools Used = | ||
Testing frameworks for use with AutoQA have already been evaluated <ref name='testing-comparison'>https://fedoraproject.org/wiki/User:Tflink/AutoQA_nose_pytest_comparison</ref> and keeping with the conclusions from that evaluation, py.test is being used to run the tests. | Testing frameworks for use with AutoQA have already been evaluated <ref name='testing-comparison'>https://fedoraproject.org/wiki/User:Tflink/AutoQA_nose_pytest_comparison</ref> and keeping with the conclusions from that evaluation, [http://doc.pytest.org py.test] is being used to run the tests. | ||
{{admon/important|Required Packages|Both {{package|python-virtualenv}} and {{package|python-pip}} are required in order to run <code>make test</code>}} | |||
Since py.test > 2.0 is still not packaged in Fedora at the time of writing, we needed to do some hacking in order to make things work. This involved the use of additional tools: | Since py.test > 2.0 is still not packaged in Fedora at the time of writing, we needed to do some hacking in order to make things work. This involved the use of additional tools: | ||
<ul> | <ul> | ||
<li> | <li>[http://www.virtualenv.org python-virtualenv]<br/> | ||
<li> | <li>[http://www.pip-installer.org python-pip]<br/> | ||
</ul> | </ul> | ||
= Process = | = Process = | ||
{{admon/note|Make Test|This process surrounding running AutoQA's self-tests need not be run manually. It can be run with <code>make test</code>}} | |||
The basic idea is to create a virtualenv with autoqa and py.test for testing to work around the fact that the currently packaged version of py.test is old. | |||
=== Virtualenv Detection === | |||
If the virtualenv is already created, there isn't much sense in creating it again. This is done in a rather simplistic fashion by looking for a directory with the name 'test_env' and if such a directory exists, a properly setup virtualenv is assumed. | |||
If the directory does not exist, a new virtualenv is created and populated with py.test (using pip) and autoqa (via symlink). | |||
=== Running the Tests === | |||
To run the tests, the virtualenv is activated and py.test is invoked. Once the tests have completed, the virtualenv is deactivated and the shell is returned to its previous state. | |||
= Gotchas = | |||
One potential snag is that this method of creating a virtualenv is uses the same python version that is running on the host system. If your setup has the same directory shared between hosts that have a different version of python, you'll have to delete the test_env directory before switching python versions. | |||
= The Future = | = The Future = | ||
This current method is a bit of a hack and once a newer version of py.test is packaged for fedora (awaiting review at the time of this writing), the process will change to be much less complicated. | |||
= References = | = References = | ||
<references> | <references> | ||
[[Category:AutoQA]] | |||
Latest revision as of 14:06, 29 June 2011
Introduction
Self tests were introduced into AutoQA with the addition of some unit tests in the 0.5.0 release. The work on the self testing facilities for AutoQA is far from finished and this is just a step in the right direction.
The structure and process for running these tests will evolve over time as we improve the testability of AutoQA and expand on the few tests that are already written.
Tools Used
Testing frameworks for use with AutoQA have already been evaluated [1] and keeping with the conclusions from that evaluation, py.test is being used to run the tests.
Since py.test > 2.0 is still not packaged in Fedora at the time of writing, we needed to do some hacking in order to make things work. This involved the use of additional tools:
Process
This process surrounding running AutoQA's self-tests need not be run manually. It can be run with
make testThe basic idea is to create a virtualenv with autoqa and py.test for testing to work around the fact that the currently packaged version of py.test is old.
Virtualenv Detection
If the virtualenv is already created, there isn't much sense in creating it again. This is done in a rather simplistic fashion by looking for a directory with the name 'test_env' and if such a directory exists, a properly setup virtualenv is assumed.
If the directory does not exist, a new virtualenv is created and populated with py.test (using pip) and autoqa (via symlink).
Running the Tests
To run the tests, the virtualenv is activated and py.test is invoked. Once the tests have completed, the virtualenv is deactivated and the shell is returned to its previous state.
Gotchas
One potential snag is that this method of creating a virtualenv is uses the same python version that is running on the host system. If your setup has the same directory shared between hosts that have a different version of python, you'll have to delete the test_env directory before switching python versions.
The Future
This current method is a bit of a hack and once a newer version of py.test is packaged for fedora (awaiting review at the time of this writing), the process will change to be much less complicated.
References
<references>