Revision as of 19:08, 3 August 2009

Introduction

This document describes the tests that will be created and used to verify the functions/components of Fedora 12.

The goals of this plan are to:

Organize the test effort
Communicate the planned tests to all relevant stake-holders for their input and approval
Serve as a base for the test planning for future Fedora 12 releases

Test Strategy

Instead of outlining all possible installation inputs and outputs, this test plan will focus on defining inputs and outputs at different stages in anaconda. This will also allow different tests to be performed independently during a single installation. For example, one may execute a kickstart delivery via HTTP, raid0 partitioning using 3 physical disks, and a minimal package installation on a para-virtualized xen guest all in single installation. Scenarios where the stages are dependent will be indicated as such in the test case.

New features of Fedora 12

As with Fedora 11, Fedora 12 will bring us some new features. The following list outlines the larger changes that affect installation. Test plans for these features will be designed/developed on each feature page.

Additional features outside the scope of testing can be found at:

Schedule/Milestones

The Fedora 12 release schedule is available at Releases/12/Schedule
Each major milestone(Beta,Preview,etc..) will demand a full regression run

Test Priority

This test plan will use a 3 tier classification for test execution priority.

Tier#1 test cases

Tier1 is intended to verify that installation is possible on common hardware using common use cases.

Verification includes:

Common boot media
Common Installation source
Installation using defaults installation options
Default Partitioning

Tier#2 test cases

Tier2 takes a step further to include more use cases. Tier2 verification consists of:

All boot media
All installation sources
All kickstart delivery methods
Some architecture specific verification

Tier#3 test cases

Tier3 captures the remaining identified use cases:

More exhaustive partitioning schemes
More complex networking scenarios
More architecture specific verification
Network device
Storage device
Upgrade testing

Test Pass/Fail Criteria

The milestone release of Fedora 12 should conform these criteria:

Entrance criteria

Trees must be generated using release engineering tools (not hand crafted)
There must be no unresolved dependencies for packages included in the installation tree
There must be no dependency conflicts for packages included in the installation tree
Any changes in composition of the installation tree are explainable by way of bugzilla

Alpha criteria

Entrance criteria have been met
All test tiers have have been executed
All tier#1 tests pass

Beta criteria

Alpha criteria have been met
All tier#2 tests pass

GA criteria

Beta criteria have been met
All test tiers must pass
Any open defects have been documented as release notes

Scope and Approach

Testing will include:

Manually execute test cases using a rawhide boot or live image media
Where possible, the use of virtualization is encouraged to allow for greater test coverage and improve failure diagnosis
Automated execution of tests is encouraged using kickstarts designed to execute specific test cases. At this time, kickstarts are not provided for every test case.

Items outside the scope of this test plan include:

Functional verification of software installed on the system
Installation from media not generated by fedora release engineering

Test Deliverables

This test plan
A test summary document for each major milestone
A list of defects filed
Any test scripts used for automation or verification

Testing Tasks

Testing will execute test cases to verify installation of Fedora 12 on different hardware platforms and gather installation test feedback.

Test Environment/Configs

For Fedora 12,test cases will be executed on these hardware platforms:

i386
ppc
x86_64

Responsibilities

Fedora QA team members are responsible for executing this test plan. Contributions from Rawhide testers and other interested parties are encouraged.

Risks and Contingencies

If no physical media is available for testing, it is acceptable to use a boot.iso or PXE boot images from a current Rawhide release. For more information, see Releases/Rawhide#Direct_Rawhide_install.

Reporting Bugs and Debugging Problems

If defects/problems are encountered, please go ahead and file the bugs following the guide below:

Reviewers

jlaska 19:08, 3 August 2009 (UTC)

References

Fedora 11 Installation Guide
Previous test plans available at Category:Test_Plans
Fedora Install test cases
Anaconda Documentation

@@ Line 1: / Line 1: @@
-{{admon/note|DRAFT|This page is currently a draft.  This test plan has not been finalized yet.  Stay tuned ...}}
 = Introduction =
 This document describes the tests that will be created and used to verify the functions/components of {{FedoraVersion|long|12}}.
@@ Line 31: / Line 29: @@
 This test plan will use a 3 '''tier''' classification for test execution priority.
-Tier1 is intended to verify that installation is possible on common hardware using common use cases.  Verification includes:
+== [[QA/TestPlans/Fedora12InstallTier1|Tier#1 test cases]] ==
+Tier1 is intended to verify that installation is possible on common hardware using common use cases.
+Verification includes:
 * Common boot media
 * Common Installation source
@@ Line 37: / Line 39: @@
 * Default Partitioning
-[[QA/TestPlans/Fedora12InstallTier1|List of tier#1 test cases]]
+== [[QA/TestPlans/Fedora12InstallTier2|Tier#2 test cases]] ==
 Tier2 takes a step further to include more use cases.  Tier2 verification consists of:
@@ Line 45: / Line 47: @@
 * Some architecture specific verification
-[[QA/TestPlans/Fedora12InstallTier2|List of tier#2 test cases]]
+== [[QA/TestPlans/Fedora12InstallTier3|Tier#3 test cases]] ==
 Tier3 captures the remaining identified use cases:
@@ Line 54: / Line 56: @@
 * Storage device
 * Upgrade testing
-[[QA/TestPlans/Fedora12InstallTier3|List of tier#3 test cases]]
 = Test Pass/Fail Criteria =
@@ Line 61: / Line 61: @@
 The milestone release of Fedora 12 should conform these criteria:
-Entrance criteria
+== Entrance criteria ==
 * Trees must be generated using release engineering tools (not hand crafted)
 * There must be no unresolved dependencies for packages included in the installation tree
@@ Line 67: / Line 67: @@
 * Any changes in composition of the installation tree are explainable by way of bugzilla
-Alpha criteria
+== Alpha criteria ==
 * Entrance criteria have been met
-* All tier1 tests have been executed
+* All test tiers have have been executed
+* All tier#1 tests pass
-Beta criteria
+== Beta criteria ==
 * Alpha criteria have been met
-* All tier1 tests pass
+* All tier#2 tests pass
-* All tier2 tests have been executed
-GA criteria
+== GA criteria ==
 * Beta criteria have been met
 * All test tiers must pass
@@ Line 84: / Line 84: @@
 Testing will include:
 * Manually execute test cases using a rawhide boot or live image media
 * Where possible, the use of virtualization is encouraged to allow for greater test coverage and improve failure diagnosis
@@ Line 93: / Line 94: @@
 = Test Deliverables =
-* This test plan
-* A test summary document for each major milestone
+* [[QA:Fedora_12_Install_Test_Plan|This test plan]]
+* [[:Category:Fedora_12_Test_Results|A test summary document for each major milestone]]
 * A list of defects filed
 * Any test scripts used for automation or verification
 = Testing Tasks =
 Testing will execute test cases to verify installation of Fedora 12 on different hardware platforms and gather installation test feedback.
 * [[QA:Fedora_12_Install_Results_Template#Test_Areas|Installation Test Cases]]
@@ Line 109: / Line 112: @@
 * ppc
 * x86_64
 = Responsibilities =
-* who's doing what
+Fedora QA team members are responsible for executing this test plan. Contributions from Rawhide testers and other interested parties are encouraged.
 = Risks and Contingencies =
-* If a particular build is uninstallable, a ''updates.img'' may be used to work around any test blockers.  For information on building and using an ''updates.img'' please refer to [[Anaconda/Updates]].
+* If no physical media is available for testing, it is acceptable to use a <code>boot.iso</code> or PXE boot images from a current Rawhide release. For more information, see [[Releases/Rawhide#Direct_Rawhide_install]].
-{{admon/note|An ''updates.img'' can't always save the day|Please note, any issues affecting stage#1 loader (networking segfaults) or kernel boot issues may not be resolvable by using an ''updates.img''.  For these issues, your defect will need to be prioritized as a test blocker so that it can be addressed in a subsequent build.}}
+= Reporting Bugs and Debugging Problems =
-* If a build is uninstallable due to a bad kernel or driver, you may be able to work around the problem.  Anaconda provides several scripts for taking a newer kernel, and using that during installation.  For details, please refer to [[http://git.fedorahosted.org/git/?p=anaconda.git;a=blob;f=scripts/upd-kernel;h=30871135eff0b4ad87bad8e0c84498a46bbf05d0;hb=HEAD| scripts/upd-kernel]]
+If defects/problems are encountered, please go ahead and file the bugs following the guide below:
+* [[Bugs_and_feature_requests|Bug Reporting Process]]
+* [[Anaconda/BugReporting|Anaconda Bug Reporting Guide]]
-= Reporting Bugs and Debugging Problems=
+= Reviewers =
-If catch some bugs,please go ahead and file the bugs following the guide below:
+* [[User:Jlaska|jlaska]] 19:08, 3 August 2009 (UTC)
-* Bug Reporting Process - [[Bugs_and_feature_requests]]
-* Anaconda Bug Reporting Guide - [[Anaconda/BugReporting]]
 = References =
@@ Line 134: / Line 139: @@
 ** [[Anaconda/Updates|How to Create and Use an updates.img]]
 ** [[Anaconda/SourceOverview|Source Code Overview]]
-** [[Anaconda/Stage1DevelopmentGuide|Anaconda Stage1 Guide]]
-** [[Anaconda/Stage2DevelopmentGuide|Anaconda Stage2 Guide]]
-= Results =
-* [[QA:Fedora 12 Alpha Install Test Results]]
 [[Category:Test Plans]]

Search

QA:Fedora 12 Install Test Plan: Difference between revisions