FWN/Beats/QualityAssurance: Difference between revisions

Revision as of 18:27, 13 July 2011

QualityAssurance

In this section, we cover the activities of the QA team^[1]. For more information on the work of the QA team and how you can get involved, see the Joining page^[2].

Contributing Writer: Adam Williamson

Test Days

The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac^[1].

↑ http://fedorahosted.org/fedora-qa/

Release criteria and validation testing

Rui He added the finished VNC installation test cases to the install testing template^[1]. Adam Williamson provided a survey comparing the Beta release criteria against the validation test cases^[2], as he previously had for the Alpha release criteria (see FWN #280), and began filing tickets to request updates to the validation tests where needed.

James Laska put into production^[3] his proposed changes to the release criteria to make them more generic (see FWN #280). James proceeded on to considering ways to implement release criteria and criteria variations specific to secondary architectures, outlining several possible ways to proceed with this^[4]. Jóhann Guðmundsson^[5], Adam Williamson^[6] and Dennis Gilmore^[7] provided feedback on his proposals, and all involved generally agreed on an approach to the problem.

Adam Williamson expanded his proposed 'base' release validation test matrix into a full proposed template page^[8], and requested feedback on the draft.

A plea to proventesters

Michael Schwendt reminded proventesters^[1] not to file positive feedback on an update for which a previous tester had provided negative feedback unless they were very sure the negative feedback was erroneous.

↑ http://lists.fedoraproject.org/pipermail/test/2011-July/101159.html

AutoQA

Having finished and released AutoQA 0.5.0, the AutoQA team was planning for the 0.6.0 development cycle^[1]. James Laska and Kamil Paral have been working on an SOP for AutoQA releases^[2].

[1] ttp://fedoraproject.org/wiki/QA

[2] ttp://fedoraproject.org/wiki/QA/Join

[3] ttp://fedorahosted.org/fedora-qa/

[4] ttp://fedorahosted.org/fedora-qa/ticket/210#comment:3

[5] ttp://fedorahosted.org/fedora-qa/ticket/151#comment:15

[6] ttp://lists.fedoraproject.org/pipermail/test/2011-July/101156.html

[7] ttp://lists.fedoraproject.org/pipermail/test/2011-July/101177.html

[8] ttp://lists.fedoraproject.org/pipermail/test/2011-July/101178.html

[9] ttp://lists.fedoraproject.org/pipermail/test/2011-July/101182.html

[10] ttp://lists.fedoraproject.org/pipermail/test/2011-July/101192.html

[11] ttp://lists.fedoraproject.org/pipermail/test/2011-July/101180.html

[12] ttp://lists.fedoraproject.org/pipermail/test/2011-July/101159.html

[13] ttp://fedorahosted.org/pipermail/autoqa-devel/2011-July/002524.html

[14] ttp://fedoraproject.org/wiki/AutoQA_Release_Process

[1]

[2]

[1]

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[1]

[1]

[2]

@@ Line 10: / Line 10: @@
 === Test Days ===
-The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>. [[User:Athmane|Athmane Madjoudj]] has been working on a proposed EC2 Test Day<ref>http://fedorahosted.org/fedora-qa/ticket/212</ref>.
+The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
 <references/>
-=== Cloud image approval ===
+=== Release criteria and validation testing ===
-At the QA group meeting of 2011-06-27<ref>http://fedoraproject.org/wiki/QA/Meetings/20110627</ref>, [[User:Rbergero|Robyn Bergeron]] emerged from a nutshell to bring up the topic of proposed pre-built Fedora images for Amazon's EC2 cloud platform<ref>http://aws.amazon.com/ec2/</ref>. These are being treated as new spins, and so they need QA approval (among other things). The group agreed to work with the Cloud SIG to draw up test procedures, and co-ordinate this through the cloud mailing list.
+[[User:Rhe|Rui He]] added the finished VNC installation test cases to the install testing template<ref>http://fedorahosted.org/fedora-qa/ticket/210#comment:3</ref>. [[User:Adamwill|Adam Williamson]] provided a survey comparing the Beta release criteria against the validation test cases<ref>http://fedorahosted.org/fedora-qa/ticket/151#comment:15</ref>, as he previously had for the Alpha release criteria (see FWN #280), and began filing tickets to request updates to the validation tests where needed.
-<references/>
+[[User:Jlaska|James Laska]] put into production<ref>http://lists.fedoraproject.org/pipermail/test/2011-July/101156.html</ref> his proposed changes to the release criteria to make them more generic (see FWN #280). James proceeded on to considering ways to implement release criteria and criteria variations specific to secondary architectures, outlining several possible ways to proceed with this<ref>http://lists.fedoraproject.org/pipermail/test/2011-July/101177.html</ref>. [[User:Johannbg|Jóhann Guðmundsson]]<ref>http://lists.fedoraproject.org/pipermail/test/2011-July/101178.html</ref>, [[User:Adamwill|Adam Williamson]]<ref>http://lists.fedoraproject.org/pipermail/test/2011-July/101182.html</ref> and [[User:Ausil|Dennis Gilmore]]<ref>http://lists.fedoraproject.org/pipermail/test/2011-July/101192.html</ref> provided feedback on his proposals, and all involved generally agreed on an approach to the problem.
-=== Test case development ===
-[[User:Jdulaney|John Dulaney]] and [[User:Rhe|Rui He]] have been working on test cases for btrfs<ref>http://fedorahosted.org/fedora-qa/ticket/209</ref>. Rui has also been working on VNC install test cases<ref>http://fedorahosted.org/fedora-qa/ticket/210</ref>.
-<references/>
-=== New proposed release validation matrices ===
-[[User:Athmane|Athmane Madjoudj]] combined the security test cases he has been working on recently into a proposed validation matrix for the Security Lab spin<ref>http://fedoraproject.org/wiki/User:Athmane/Draft_Security_Lab_validation_matrix</ref> and asked the group for feedback on it<ref>http://fedorahosted.org/fedora-qa/ticket/207</ref>. [[User:Adamwill|Adam Williamson]] proposed a new validation matrix<ref>http://fedoraproject.org/wiki/User:Adamwill/Draft_base_validation_matrix</ref> for validation tests which do not fit under the existing 'install' or 'desktop' matrices, for now referring to it as 'base', and asked for feedback<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/101035.html</ref>.
-<references/>
-=== Ensuring consistency of validation tests and release criteria ===
-[[User:Adamwill|Adam Williamson]] began a survey to check the consistency of the release validation tests and the release criteria, to ensure tests were present for every criterion<ref>http://fedorahosted.org/fedora-qa/ticket/151</ref>. Together with [[User:Rhe|Rui He]] and [[User:Athmane|Athmane Madjoudj]], he created new test cases and adjusted existing ones to ensure consistency with the Alpha release criteria.
+[[User:Adamwill|Adam Williamson]] expanded his proposed 'base' release validation test matrix into a full proposed template page<ref>http://lists.fedoraproject.org/pipermail/test/2011-July/101180.html</ref>, and requested feedback on the draft.
 <references/>
-=== Making release criteria more generic ===
+=== A plea to proventesters ===
-[[User:Jlaska|James Laska]] proposed several changes to the release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100984.html</ref>, intended to make them more generic and hence applicable to secondary architectures. [[User:Adamwill|Adam Williamson]] generally liked the proposal but made some comments and asked some questions about some of them<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100989.html</ref>.
+[[User:Mschwendt|Michael Schwendt]] reminded proventesters<ref>http://lists.fedoraproject.org/pipermail/test/2011-July/101159.html</ref> not to file positive feedback on an update for which a previous tester had provided negative feedback unless they were very sure the negative feedback was erroneous.
 <references/>
@@ Line 46: / Line 32: @@
 === AutoQA ===
-The team was in the final run-up to the release of AutoQA 0.5.0, which would include the previously-discussed 'pretty output' and spam reduction patches. The release of 0.5.0 duly happened on 2011-06-29<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002496.html</ref>. Lucas Rodrigues announced the release of Autotest 0.13.0<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002454.html</ref>, with packages available for testing in the autoqa-testing repository.
+Having finished and released AutoQA 0.5.0, the AutoQA team was planning for the 0.6.0 development cycle<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-July/002524.html</ref>. [[User:Jlaska|James Laska]] and [[User:Kparal|Kamil Paral]] have been working on an SOP for AutoQA releases<ref>http://fedoraproject.org/wiki/AutoQA_Release_Process</ref>.
 <references/>

Search