QA/Beaker

Beaker

Beaker is an adaptation of Red Hat's RHTS automated test system for Fedora (and the Free Software community in general). See ["QA/RHTS"] for more info about RHTS.

Where we are

As mentioned elsewhere, Red Hat QE has released the following pieces of their test system:

• Documentation and APIs for tests, test running, test result reporting, etc.
• Automated tests which use those APIs
• A personal test execution engine (so individual testers can run tests on their systems)

Where we're going

Now that we have a suggested format / API for tests, and ways to run those tests, the next obvious step is collecting test results and displaying them in an attractive, easy-to-use way. So here's what's next:

• Streamlined test-writing
• Writing RHTS tests is not hard, but it could be easier. We're considering creating a simplified test format for Beaker.
• A few design goals:

1. Tests should not be responsible for anything other than doing testing and emitting output on stdout, stderr, and maybe writing some files (typically to an agreed-upon place) 1. Tests should run without needing a special test environment. 1. Tests should be able to be written in any language the author chooses. 1. Tests should be verbose about what they are doing and how they are doing it.

• A common test result reporting format.
• RHTS has an XML-RPC API for reporting test results.
• There are other test result schema out there - for example, Spikesource has their OSL-licensed Test Results Publication Interface, as defined here: http://developer.spikesource.com/wiki/index.php/Test_Results_Publication_Interface
• A test-runner
• Once we've got tests and a common reporting format, we need a tool that will take the test output and turn it into output in the common format.
• A results aggregator - a place to send, display, and query test results.
• This will take files using the common test result format and display them to users, testers, and other interested parties.

Here is a tentative roadmap/schedule for some important Beaker achievements:

Beaker Roadmap
Date	Task	Status/Notes
June 30	Red Hat releases sample tests, simple test execution engine	OK, test release ongoing
August 30	Test API documentation available	https://testing.108.redhat.com/test-writing-api/
January 24	RHTS test-env packages in Extras	Under review
	Test runner and common results format
	First prototype results server
	Completed results server