User:Roshi/QA/Cloud Docs

Overview

This is a collection of docs related to testing of the Cloud product. As these get solidified they'll go in more official places than my wikispace. Currently on the tick-list:

• Identify Relevant Release Criteria
• Draft a Test Plan
• Draft a Test Matrix for each Milestone

Relevant Release Criteria

This is not meant to replace the existing RC pages. It was just easier for me to parse after taking out all the parts that don't directly deal with Cloud images. All release criteria referenced in the test plan will correlate to the actual Fedora 21 Fedora Release Criteria.

• Cloud Alpha Release Criteria
• Cloud Beta Release Criteria
• Cloud Final Release Criteria

Test Overview

• Initial Draft
• Running a cloud instance locally

Test Plan

• Initial Draft

Test Matrix

Traditionally we've had a couple validation matrices for a particular release (Base, Installation and Desktop). However, for cloud I think a single matrix should suffice for our testing needs. It's basically a fork of the QA:Base_validation_results_template with cloud testcases and milestones added in.

• Cloud Validation Matrix

Known Testcases

This is the list of testcases that I've found related to cloud images. Some of these need cleaning and assessed for if they can be automated.

• Ideas on what to test

AWS

• http://fedoraproject.org/wiki/QA:Testcase_EC2_AMI_Validation

Openstack

• http://fedoraproject.org/wiki/QA:Testcase_OpenStack_Image_Validation
• http://fedoraproject.org/wiki/QA:Testcase_OpenStack_ceilometer_install
• http://fedoraproject.org/wiki/QA:Testcase_OpenStack_floating_IPs
• http://fedoraproject.org/wiki/QA:Testcase_launch_an_instance_on_OpenStack
• http://fedoraproject.org/wiki/QA:Testcase_register_images_with_OpenStack

Atomic/Docker

Atomic and Docker images won't be release blocking for F21. However, the hope is to have them be proper products for F22. Here is where we'll document the process to use when F22 gets here. I see this happening in 3 phases: Plan/Document, Draft, Integrate.

Plan/Document

First is to figure out what things need to be tested for Atomic and Docker images. We also have to look at the existing Release Criteria and draft any changes they might need in order to accommodate Atomic and Docker images. Use the two pages below to add ideas for testcases.

• Atomic Test Ideas
• Docker Test Ideas

Draft

Once the needed tests are decided on, we need to write the testcases and Release Criteria out here on the wiki. From here Atomic/Cloud/QA can discuss the proposed changes. After we have testcases written, cross-referenced with the Release Criteria and discussed, work can start on automating the created test cases.

Integrate

The last step is to work with QA and figure out the best means to support the added changes for F22. Something that will need to be hashed out is manpower (who will be doing the testing, who all shows up to blocker review meetings), and I'm sure other things I haven't thought of yet.