No edit summary |
|||
| Line 32: | Line 32: | ||
! Alpha test cases !! Beta test cases !! Final test cases | ! Alpha test cases !! Beta test cases !! Final test cases | ||
|- | |- | ||
| Alpha (formerly tier#1) priority tests are intended to verify that | | Alpha (formerly tier#1) priority tests are intended to verify that booting the image is possible on common cloud providers (EC2, Openshift). These tests also attempt to validate [[User:Roshi/QA/Cloud_Docs/Cloud_Alpha_Release_Criteria|Alpha Release Requirements]]. | ||
| Beta (formerly tier#2) priority tests take a step further to include additional use cases | | Beta (formerly tier#2) priority tests take a step further to include additional use cases. These tests also attempt to validate [[User:Roshi/QA/Cloud_Docs/Cloud_Beta_Release_Criteria|Beta Release Requirements]]. | ||
| Final (formerly tier#3) priority tests capture all remaining use cases and | | Final (formerly tier#3) priority tests capture all remaining use cases and functionality checks. These tests also attempt to validate [[User:Roshi/QA/Cloud_Docs/Cloud_Final_Release_Criteria|Final Release Requirements]]. | ||
|- | |- | ||
| Verification consists of: | | Verification consists of: | ||
| Verification consists of: | | Verification consists of: | ||
| Verification consists of: | | Verification consists of: | ||
|} | |} | ||
Revision as of 02:05, 24 April 2014
History
Created by Roshi on 22 April 2014
Introduction
With the advent of different Fedora Products for Fedora 21 there is a need for a specific test plan for each product. Historically, Release Validation and all Testing of Fedora releases was handled by QA - but now testing and release validation will be largely handled by the specific Working Group.
The goals of this plan are to:
- Organize the test effort
- Communicate the planned tests to all relevant stake-holders for their input and approval
- Serve as a base for the test planning for future Fedora Cloud product releases
Test Strategy
Testing and Release Validation for the Cloud product will follow the same pattern Fedora 20 followed. There will be three milestones - each milestone will have a related set of Release Criteria and Validation Matrix for the release.
Milestones
- Alpha
- Beta
- Final
Schedule/Milestones
Test Priority
This test plan prioritizes tests according to the major release milestones for Fedora 21, including the Alpha, Beta and Final release milestones. All test cases are intended for execution at every milestone. However, priority should be given to tests specific to the milestone under test.
| Alpha test cases | Beta test cases | Final test cases |
|---|---|---|
| Alpha (formerly tier#1) priority tests are intended to verify that booting the image is possible on common cloud providers (EC2, Openshift). These tests also attempt to validate Alpha Release Requirements. | Beta (formerly tier#2) priority tests take a step further to include additional use cases. These tests also attempt to validate Beta Release Requirements. | Final (formerly tier#3) priority tests capture all remaining use cases and functionality checks. These tests also attempt to validate Final Release Requirements. |
| Verification consists of: | Verification consists of: | Verification consists of: |
Test Pass/Fail Criteria
The milestone release of Fedora 17 should conform these criteria:
Entrance criteria
- Trees must be generated using release engineering tools (not hand crafted)
- There must be no unresolved dependencies for packages included in the installation tree
- There must be no dependency conflicts for packages included in the installation tree
- Any changes in composition of the installation tree are explainable by way of bugzilla
Milestone specific criteria
Scope and Approach
For the cloud image, the following areas will be tested:
- Boot process
- Initialization
- Post-Boot actions
- Virtualization
In addition to those aspects of the image being tested, any bugs marked as a blocker for the current milestone must be addressed and tested.
Testing will include:
- Manually executed test cases using DVD,
boot.iso, PXE or live image media - Automatically executed test cases via the AutoQA system. For more information about automatic testing, please see Is Anaconda Broken Proposal and the install automation roadmap.
Test Deliverables
- This test plan
- Test summary documents for each major milestone of F16: Category:Fedora_17_Test_Results
- A list of defects filed
- Any test scripts used for automation or verification
Testing Tasks
Testing will execute test cases to verify installation of Fedora 17 on different hardware platforms and gather installation test feedback.
- Manual installation Test Cases
- Auto-installation Test Cases
- Instructions for adding test result page
Test Environment/Configs
For Fedora 17, test cases will be executed on the primary supported hardware platforms. This includes:
- i386
- x86_64
Responsibilities
Fedora QA team members are responsible for executing this test plan. Contributions from Branched testers and other interested parties are encouraged.
Risks and Contingencies
If new physical media are provided for an already inprogress test run, a new test run must be initiated. Test results from the previous run may be carried forward to the new test run if they are not affected by the changes introduced by the new physical media.
Reporting Bugs and Debugging Problems
If defects/problems are encountered, please go ahead and file the bugs following the guide below:
Reviewers
References
- Installation Validation Testing Introduction Page
- Fedora 16 Installation Guide
- Fedora Install test cases
- Previous test plans available at Category:Test_Plans
- Anaconda Documentation