From Fedora Project Wiki

< FWN‎ | Beats

(create fwn 279 qa beat)
(create fwn 280 qa beat)
Line 9: Line 9:


=== Test Days ===
=== Test Days ===
A special Test Day<ref>http://fedoraproject.org/wiki/QA/Test_Day:2011-06-08_IPv6</ref> was held on Wednesday 2011-06-08 for World IPv6 Day<ref>http://www.worldipv6day.org/</ref>, thanks to Linda Wang. Despite the complexity involved in implementing an IPv6 setup for testing, a good group of testers were able to run through the various tests and identify some bugs which have already received developer attention.


The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
Line 16: Line 14:
<references/>
<references/>


=== Security spin testing ===
=== Cloud image approval ===
 
[[User:Athmane|Athmane Madjoudj]] announced some testing he had done on the Fedora 15 security spin<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100484.html</ref>. [[User:Adamwill|Adam Williamson]] thanked him and asked if he had contacted the security spin authors about it<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100485.html</ref>, and Athmane replied that he would. Later, Athmane announced that he had added some more tests<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100496.html</ref>, and Adam suggested writing them up as test cases<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100497.html</ref>. Athmane did this<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100511.html</ref>, and then continued to add more test cases regularly. In the course of this work, Athmane noticed and helped to address some problems with the rendering of tags and templates in the wiki<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100668.html</ref> <ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100763.html</ref>.
 
<references/>
 
=== Release criteria revisions ===
 
[[User:Adamwill|Adam Williamson]] continued with revisions of the release criteria. He announced that the wording 'release-blocking desktops' to describe the desktops that are capable of blocking release had been generally well received, so he had updated the criteria to use this wording<ref>http://lists.fedoraproject.org/pipermail/test/2011-May/100646.html</ref>.
 
<references/>
 
=== Installer validation test revisions ===


[[User:Rhe|Rui He]] reported that she had refined the various installation source tests that form part of the installation validation matrix by creating some new tests and renaming and adjusting others<ref>http://fedorahosted.org/fedora-qa/ticket/173#comment:7</ref>. [[User:Jlaska|James Laska]] followed up with some questions and comments<ref>http://fedorahosted.org/fedora-qa/ticket/173#comment:8</ref>, and Rui He continued with further improvements.
At the QA group meeting of 2011-06-27<ref>http://fedoraproject.org/wiki/QA/Meetings/20110627</ref>, [[User:Rbergero|Robyn Bergeron]] emerged from a nutshell to bring up the topic of proposed pre-built Fedora images for Amazon's EC2 cloud platform<ref>http://aws.amazon.com/ec2/</ref>. These are being treated as new spins, and so they need QA approval (among other things). The group agreed to work with the Cloud SIG to draw up test procedures, and co-ordinate this through the cloud mailing list.


<references/>
<references/>


=== Overly similar application names ===
=== New proposed release validation matrices ===


Vitezslav Humpa started a discussion and made some proposals around the issue of applications with very similar names and icons in the system menus<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100684.html</ref>. This resulted in an active and productive debate across several teams about the best way to move forward in addressing the problem. In the end it was agreed that it would be best and fastest to work on a case-by-case basis through the most commonly-encountered name collisions.
[[User:Athmane|Athmane Madjoudj]] combined the security test cases he has been working on recently into a proposed validation matrix for the Security Lab spin<ref>http://fedoraproject.org/wiki/User:Athmane/Draft_Security_Lab_validation_matrix</ref> and asked the group for feedback on it<ref>http://fedorahosted.org/fedora-qa/ticket/207</ref>. [[User:Adamwill|Adam Williamson]] proposed a new validation matrix<ref>http://fedoraproject.org/wiki/User:Adamwill/Draft_base_validation_matrix</ref> for validation tests which do not fit under the existing 'install' or 'desktop' matrices, for now referring to it as 'base', and asked for feedback<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/101035.html</ref>.


<references/>
<references/>


=== Draft btrfs test case ===
=== Ensuring consistency of validation tests and release criteria ===


[[User:Jdulaney|John Dulaney]] announced that he had been working on a btrfs test case in advance of Fedora 16, where it is likely to be the default filesystem, and that he had a draft available for review<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100749.html</ref>. [[User:Rhe|Rui He]], [[User:Sundaram|Rahul Sundaram]] and JB replied with comments and suggestions.
[[User:Adamwill|Adam Williamson]] began a survey to check the consistency of the release validation tests and the release criteria, to ensure tests were present for every criterion<ref>http://fedorahosted.org/fedora-qa/ticket/151</ref>. Together with [[User:Rhe|Rui He]] and [[User:Athmane|Athmane Madjoudj]], he created new test cases and adjusted existing ones to ensure consistency with the Alpha release criteria.


<references/>
<references/>


=== Fedora 15 QA retrospective ===
=== Making release criteria more generic ===


[[User:Jlaska|James Laska]] announced that he had completed the Fedora 15 QA retrospective wiki page, and drafted a set of recommendations for review by the group. He asked for feedback on the retrospective and the recommendations<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100904.html</ref>.
[[User:Jlaska|James Laska]] proposed several changes to the release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100984.html</ref>, intended to make them more generic and hence applicable to secondary architectures. [[User:Adamwill|Adam Williamson]] generally liked the proposal but made some comments and asked some questions about some of them<ref>http://lists.fedoraproject.org/pipermail/test/2011-June/100989.html</ref>.  


<references/>
<references/>
Line 54: Line 40:
=== AutoQA ===
=== AutoQA ===


There were two big AutoQA developments since the last newsletter. One was the so-called 'pretty patch', which improved the layout and legibility of AutoQA results, particularly the dependency check test. It was submitted to the mailing list on 2011-06-06<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002344.html</ref> by [[User:Kparal|Kamil Paral]] and pushed on 2011-06-10<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002385.html</ref>. The other was a patch from [[User:Tflink|Tim Flink]] to reduce the volume of messages sent out by AutoQA, notably by not sending emails to maintainers when the tests are entirely successful. This patch was submitted on 2011-06-06<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002346.html</ref> and merged on 2011-06-09<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002372.html</ref>.
The team was in the final run-up to the release of AutoQA 0.5.0, which would include the previously-discussed 'pretty output' and spam reduction patches. Lucas Rodrigues announced the release of Autotest 0.13.0<ref>http://fedorahosted.org/pipermail/autoqa-devel/2011-June/002454.html</ref>, with packages available for testing in the autoqa-testing repository.


<references/>
<references/>

Revision as of 20:29, 30 June 2011

QualityAssurance

In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].

Contributing Writer: Adam Williamson

Test Days

The Fedora 15 Test Day track is now finished, and the main Fedora 16 Test Day track has not yet started. If you would like to propose a main track Test Day for the Fedora 16 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[1].

Cloud image approval

At the QA group meeting of 2011-06-27[1], Robyn Bergeron emerged from a nutshell to bring up the topic of proposed pre-built Fedora images for Amazon's EC2 cloud platform[2]. These are being treated as new spins, and so they need QA approval (among other things). The group agreed to work with the Cloud SIG to draw up test procedures, and co-ordinate this through the cloud mailing list.

New proposed release validation matrices

Athmane Madjoudj combined the security test cases he has been working on recently into a proposed validation matrix for the Security Lab spin[1] and asked the group for feedback on it[2]. Adam Williamson proposed a new validation matrix[3] for validation tests which do not fit under the existing 'install' or 'desktop' matrices, for now referring to it as 'base', and asked for feedback[4].

Ensuring consistency of validation tests and release criteria

Adam Williamson began a survey to check the consistency of the release validation tests and the release criteria, to ensure tests were present for every criterion[1]. Together with Rui He and Athmane Madjoudj, he created new test cases and adjusted existing ones to ensure consistency with the Alpha release criteria.

Making release criteria more generic

James Laska proposed several changes to the release criteria[1], intended to make them more generic and hence applicable to secondary architectures. Adam Williamson generally liked the proposal but made some comments and asked some questions about some of them[2].

AutoQA

The team was in the final run-up to the release of AutoQA 0.5.0, which would include the previously-discussed 'pretty output' and spam reduction patches. Lucas Rodrigues announced the release of Autotest 0.13.0[1], with packages available for testing in the autoqa-testing repository.