|
|
| (3 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| = Beaker = | | = Beaker = |
|
| |
|
| Beaker is an adaptation of Red Hat's RHTS automated test system for Fedora (and the Free Software community in general). See ["QA/RHTS"] for more info about RHTS. | | Beaker is an automated test system created by Red Hat that can handle full stack hardware and software integration testing for Fedora, Red Hat Enterprise Linux, CentOS and derivatives. |
| | |
| | Project home page: http://beaker-project.org/ |
|
| |
|
| === Where we are === | | === Where we are === |
|
| |
|
| As mentioned elsewhere, Red Hat QE has released the following pieces of their test system:
| | While Red Hat released Beaker as an open source project several years ago, none of the regression tests were released at that point in time. |
| | |
| * Documentation and APIs for tests, test running, test result reporting, etc.
| |
| * Automated tests which use those APIs
| |
| * A personal test execution engine (so individual testers can run tests on their systems)
| |
| | |
| === Where we're going ===
| |
| | |
| Now that we have a suggested format / API for tests, and ways to run those tests, the next obvious step is collecting test results and displaying them in an attractive, easy-to-use way. So here's what's next:
| |
| | |
| * Streamlined test-writing
| |
| * Writing RHTS tests is not hard, but it could be easier. We're considering creating a simplified test format for Beaker.
| |
| * A few design goals:
| |
| 1. Tests should not be responsible for anything other than doing testing and emitting output on stdout, stderr, and maybe writing some files (typically to an agreed-upon place)
| |
| 1. Tests should run without needing a special test environment.
| |
| 1. Tests should be able to be written in any language the author chooses.
| |
| 1. Tests should be verbose about what they are doing and how they are doing it.
| |
| <!-- should link to QA/BeakerTestFormat or something
| |
| -->
| |
| * A common test result reporting format.
| |
| * RHTS has an XML-RPC API for reporting test results.
| |
| * There are other test result schema out there - for example, Spikesource has their OSL-licensed Test Results Publication Interface, as defined here: http://developer.spikesource.com/wiki/index.php/Test_Results_Publication_Interface
| |
| * A test-runner
| |
| * Once we've got tests and a common reporting format, we need a tool that will take the test output and turn it into output in the common format.
| |
| * A results aggregator - a place to send, display, and query test results.
| |
| * This will take files using the common test result format and display them to users, testers, and other interested parties.
| |
|
| |
|
| Here is a tentative roadmap/schedule for some important Beaker achievements:
| | This is now changing, with the essential installer regression tests being made available for use in Fedora as well: http://atodorov.org/blog/2013/11/19/open-source-quality-assurance-infrastructure-for-fedora-qa/ |
|
| |
|
| {| border="1"
| | The Fedora Beaker instance is at: http://beaker.qa.fedoraproject.org/ |
| |- ||||style="color: white; background-color: #3070c0; font-weight: bold"
| |
| |Beaker Roadmap
| |
| |- style="color: white; background-color: #153560; font-weight: bold"
| |
| |Date||Task||Status/Notes
| |
| |- style="color: white; background-color: green"
| |
| |June 30||Red Hat releases sample tests, simple test execution engine||OK, test release ongoing
| |
| |- style="color: white; background-color: green"
| |
| |August 30||Test API documentation available||https://testing.108.redhat.com/test-writing-api/
| |
| |- style="color: white; background-color: #040"
| |
| |January 24||RHTS test-env packages in Extras||Under review
| |
| |-
| |
| | ||Test runner and common results format||
| |
| |-
| |
| | ||First prototype results server||
| |
| |-
| |
| | ||Completed results server||
| |
