Beaker
Beaker is an adaptation of Red Hat's RHTS automated test system for Fedora (and the Free Software community in general). See ["QA/RHTS"] for more info about RHTS.
Where we are
As mentioned elsewhere, Red Hat QE has released the following pieces of their test system:
- Documentation and APIs for tests, test running, test result reporting, etc.
- Automated tests which use those APIs
- A personal test execution engine (so individual testers can run tests on their systems)
Where we're going
Now that we have a suggested format / API for tests, and ways to run those tests, the next obvious step is collecting test results and displaying them in an attractive, easy-to-use way. So here's what's next:
- Streamlined test-writing
- Writing RHTS tests is not hard, but it could be easier. We're considering creating a simplified test format for Beaker.
- A few design goals:
1. Tests should not be responsible for anything other than doing testing and emitting output on stdout, stderr, and maybe writing some files (typically to an agreed-upon place) 1. Tests should run without needing a special test environment. 1. Tests should be able to be written in any language the author chooses. 1. Tests should be verbose about what they are doing and how they are doing it.
- A common test result reporting format.
- RHTS has an XML-RPC API for reporting test results.
- There are other test result schema out there - for example, Spikesource has their OSL-licensed Test Results Publication Interface, as defined here: http://developer.spikesource.com/wiki/index.php/Test_Results_Publication_Interface
- A test-runner
- Once we've got tests and a common reporting format, we need a tool that will take the test output and turn it into output in the common format.
- A results aggregator - a place to send, display, and query test results.
- This will take files using the common test result format and display them to users, testers, and other interested parties.
Here is a tentative roadmap/schedule for some important Beaker achievements:
| Beaker Roadmap | ||
| Date | Task | Status/Notes |
| June 30 | Red Hat releases sample tests, simple test execution engine | OK, test release ongoing |
| August 30 | Test API documentation available | https://testing.108.redhat.com/test-writing-api/ |
| January 24 | RHTS test-env packages in Extras | Under review |
| Test runner and common results format | ||
| First prototype results server | ||
| Completed results server |