From Fedora Project Wiki

(Save page-in-progress)
(Complete first draft)
Line 1: Line 1:
= Revision history =
= Revision history =


First draft: [[User:Wwoods|WillWoods]] 19:46, 17 June 2009 (UTC)
First draft: [[User:Wwoods|WillWoods]] 20:54, 17 June 2009 (UTC)


= Introduction =
= Introduction =
Line 17: Line 17:
TBD. No clear priorities exist yet.
TBD. No clear priorities exist yet.
-->
-->
= Scope =
= Scope =


Line 41: Line 40:
* stage2 is able to detect the presence of disks attached to most common controllers
* stage2 is able to detect the presence of disks attached to most common controllers
* Kernel/X is able to set up common display configurations for at least 2 out of 3 of the most common video drivers (intel, nouveau, radeon)
* Kernel/X is able to set up common display configurations for at least 2 out of 3 of the most common video drivers (intel, nouveau, radeon)
'''TODO FINISH BELOW HERE'''


= Test Deliverables =
= Test Deliverables =


Describe what the consumables will be from testing.  Examples include:
This test plan should produce:
* This test plan
* A summary report on whether Rawhide is broken or not
* A test summary document for each major milestone
* Bug reports for broken dependencies / missing files / etc.
* A list of defects filed
* Any test scripts used for automation or verification


= Test Cases  =
= Test Cases  =
Using the level of detail required by this test plan, outline test areas and test cases to be exercised.
 
''TODO: create these test cases''
 
# Repodata validity
# comps.xml validity
# Core package existence
# Core package dependency closure
# Installer image existence
# Kernel boot
# Anaconda loader fetching stage2
# Anaconda stage2 disk probe
# X startup


= Test Environment =
= Test Environment =
Describe the environment tests will be executed in
 
* Tests will run on a host which has the Rawhide tree accessible on the local filesystem. It may be network-mounted.


= Responsibilities =
= Responsibilities =
High-level detail of who is accountable for different phases of testing.
 
Fedora QA team members are responsible for executing this test plan. Contributions from Rawhide testers and other interested parties are encouraged.


= Schedule =
= Schedule =


Outline the scheduled milestones.
Ideally this test plan should be run for ''every new Rawhide tree'' - i.e. daily.


= Risks =
= Risks =


Given the current scope and test priority, what are some potential risks. Are there any other contingencies should problems arrise?  How will issues be dealt with when they surface.
* Testing kernel boot and X startup is currently hard to automate.
* We don't have a good definition for "Core packages" - personal opinions may vary wildly.


= Reviewers =
= Reviewers =


 
<!-- Add a line like this:
* ~~~~
This will automatically be expanded to your username and timestamp. Preview to check it. -->


= References =
= References =


<references/>
<references/>

Revision as of 20:54, 17 June 2009

Revision history

First draft: WillWoods 20:54, 17 June 2009 (UTC)

Introduction

This test plan documents the process used to check the basic requirements for a Rawhide tree to be acceptable for further testing. It aims to check whether Rawhide is installable, usable as a package repo for updating, and whether critical packages are present and functional.

In short, this is how we decide if Rawhide is broken or not.

Test Strategy

There are three main components here: Repo sanity, Installability, and Basic Functionality. These three categories can be tested mostly independent of one another.

Scope

This plan seeks to answer three basic questions:

  1. Can current Rawhide users update their systems using this repo?
  2. Can this Rawhide tree be installed?
  3. Does the basic system (the "critical path") work as expected for simple testing?

It is not intended to be an exhaustive test of any part of the system.

Test Pass/Fail Criteria

Rawhide will be considered Good if all of the following conditions are met:

  • Contains valid yum metadata (repodata)
  • Contains key packages (kernel, glibc, coreutils)
  • No unresolved dependencies in critical packages
  • comps.xml exists and is valid
  • installer images (kernel, initrd, install.img, and boot.iso) exist
  • kernel boots on most machines of the primary architectures
  • initrd is able to find stage2 (install.img) by at least one method (network, local CD)
  • stage2 is able to detect the presence of disks attached to most common controllers
  • Kernel/X is able to set up common display configurations for at least 2 out of 3 of the most common video drivers (intel, nouveau, radeon)

Test Deliverables

This test plan should produce:

  • A summary report on whether Rawhide is broken or not
  • Bug reports for broken dependencies / missing files / etc.

Test Cases

TODO: create these test cases

  1. Repodata validity
  2. comps.xml validity
  3. Core package existence
  4. Core package dependency closure
  5. Installer image existence
  6. Kernel boot
  7. Anaconda loader fetching stage2
  8. Anaconda stage2 disk probe
  9. X startup

Test Environment

  • Tests will run on a host which has the Rawhide tree accessible on the local filesystem. It may be network-mounted.

Responsibilities

Fedora QA team members are responsible for executing this test plan. Contributions from Rawhide testers and other interested parties are encouraged.

Schedule

Ideally this test plan should be run for every new Rawhide tree - i.e. daily.

Risks

  • Testing kernel boot and X startup is currently hard to automate.
  • We don't have a good definition for "Core packages" - personal opinions may vary wildly.

Reviewers

References

Retrieved from "https://fedoraproject.org/w/index.php?title=QA:Rawhide_Acceptance_Test_Plan&oldid=108895"