| (One intermediate revision by the same user not shown) | |||
| Line 26: | Line 26: | ||
== Scope == | == Scope == | ||
With the hard work in anaconda already having been done (reorganizing into a proper Python module, making the storage code externally usable, fixing the live environment) all that is left to do is write test cases which is just in autoqa. As | With the hard work in anaconda already having been done (reorganizing into a proper Python module, making the storage code externally usable, fixing the live environment) and the framework integrated into autoqa, all that is left to do is write test cases which is just in autoqa. As of Jan 18, 2011 there is not yet an autoqa build with the storage tests included but that will be taken care of with the next build. Bugs may be discovered in anaconda or pykickstart but that is just standard bug fixing work. | ||
== Test Plan == | == Test Plan == | ||
Run the storage tests as part of the autoqa, check the results. If any tests fail because of bugs in the cases or framework itself, fix them. For all others, it's a bug in anaconda itself. | |||
== User Experience == | == User Experience == | ||
Latest revision as of 18:17, 18 January 2011
Finish storage testing frameworks and put to use
Summary
Finalize storage testing frameworks and integrate in to a continual test system as well as QE's larger testing frameworks.
Owner
- Name: Chris Lumens
Current status
- Targeted release: Fedora 15
- Last updated: 2011-01-18
- Percentage of completion: 75% - integration complete, need to add more tests
Detailed Description
I have developed a test framework that can be hooked into autoqa and exercises the storage code external to the installer environment. Tests can be chained together so that the output of one test (a set of disks) is the input of another test. The entire framework runs under virtualization. This set of features means tests can be run completely without user intervention (making it perfect to run weekly, nightly, at build time, or even commit time though that would be very slow), from a known preexisting disk state (making failures reproducible), and without damaging existing hardware.
The real strength of this framework is first that we can automate the entirety of the storage section of the release test matrix, and second that we can develop regression tests from bug reports given enough detail. Making use of this strength requires writing a lot of tests, though.
Benefit to Fedora
Automated storage testing means we can catch storage problems faster, create regression tests, and reduce QA workload by getting them out of the storage testing business. This should directly result in a better installer (due to fewer bugs) and indirectly to better releases (due to QA being able to focus on other things).
Scope
With the hard work in anaconda already having been done (reorganizing into a proper Python module, making the storage code externally usable, fixing the live environment) and the framework integrated into autoqa, all that is left to do is write test cases which is just in autoqa. As of Jan 18, 2011 there is not yet an autoqa build with the storage tests included but that will be taken care of with the next build. Bugs may be discovered in anaconda or pykickstart but that is just standard bug fixing work.
Test Plan
Run the storage tests as part of the autoqa, check the results. If any tests fail because of bugs in the cases or framework itself, fix them. For all others, it's a bug in anaconda itself.