(s/https/http/) |
(create fwn 249 qa beat) |
||
| Line 3: | Line 3: | ||
In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>. For more information on the work of the QA team and how you can get involved, see the Joining page<ref>http://fedoraproject.org/wiki/QA/Join</ref>. | In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>. For more information on the work of the QA team and how you can get involved, see the Joining page<ref>http://fedoraproject.org/wiki/QA/Join</ref>. | ||
Contributing Writer: [[User:Adamwill|Adam Williamson]] | Contributing Writer: [[User:Adamwill|Adam Williamson]] | ||
| Line 12: | Line 10: | ||
=== Test Days === | === Test Days === | ||
The last Fedora 14 Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-10-14_OpenLDAP/NSS</ref> on 2010-10-14 was on the use of OpenLDAP with the NSS security library - the use of NSS with OpenLDAP is new in Fedora 14, replacing the previous use of OpenSSL. [[User:Kparal|Kamil Paral]] provided a recap to the list<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094698.html</ref>. He noted that "the participation was little low, but it was somehow expected, because this was a non-trivial topic" and that the event discovered three bugs and confirmed the rest of the functionality worked. | |||
If you would like to propose a main track Test Day for the Fedora 15 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>. | |||
<references/> | <references/> | ||
=== Fedora 14 | === Fedora 14 Final testing === | ||
The group | The QA group put together a great team effort to perform comprehensive and efficient validation testing on the Fedora 14 final release. The Test Compose was announced on 2010-10-13<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094596.html</ref>, and the installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_TC1_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_TC1_Desktop</ref> test matrices were completed quickly, with results coming in from many different group members. The testing identified several blocker bugs, which were all resolved in time for the release of the Release Candidate on 2010-10-21<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094893.html</ref>. Once again, the group came together to perform the installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_RC1_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_RC1_Desktop</ref> validation testing, which was mostly completed by the weekend. No further blocker bugs were discovered (as outlined by [[User:Rhe|Rui He]] in the RC1 testing recap<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094998.html</ref>), and so the QA group was able to join with release engineering and development to sign off on the release of RC1 as Fedora 14 Final at the go/no-go meeting of 2010-10-26<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-10-26/fedora-meeting.2010-10-26-20.59.log.html</ref>. [[User:Adamwill|Adam Williamson]] thanked all of the large number of community members who contributed to the validation testing in a blog post<ref>http://www.happyassassin.net/2010/10/26/fedora-14-goes-gold/</ref>. | ||
<references/> | <references/> | ||
=== | === Reporting bugs from downstream distributions === | ||
[[User: | [[User:Tk009|Edward Kirk]] reported that the lead developer of Fusion Linux, a distribution based on Fedora, was suggesting users file bugs in Fedora Bugzilla<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094830.html</ref>. A discussion ensued on whether it was correct for users of distributions downstream of Fedora to report bugs directly to Fedora's bug tracker. [[User:Johannbg|Jóhann Guðmundsson]] felt strongly that it was not appropriate, and such distributions should have their own bug trackers<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094844.html</ref>. [[User:Mschwendt|Michael Schwendt]] pointed out that Fedora's abrt does not currently make it easy for downstream distributions to modify it to report crashes to a different bug tracking system<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094848.html</ref>. [[User:Adamwill|Adam Williamson]] thought it made sense for bugs in Fedora packages which are present unchanged in Fusion Linux to be reported to Fedora's bug tracker, in the same way Fedora bug reports are often sent upstream to GNOME or KDE<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094852.html</ref>. The Fusion Linux developer, [[User:Valent|Valent Turkovic]], said that his plan was for Fusion Linux developers to check reports of bugs and ask the user to file them in Fedora Bugzilla if the package was unchanged from Fedora, RPM Fusion Bugzilla if the package came from there, or with Fusion Linux's developers if the bug related to Fusion Linux-specific customizations. | ||
<references/> | <references/> | ||
| Line 34: | Line 30: | ||
=== Release criteria === | === Release criteria === | ||
[[User: | [[User:Adamwill|Adam Williamson]] proposed a new release criterion<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094785.html</ref> requiring the final release notes from the Documentation team be present in packaged form in the release repository (at Final release stage). [[User:Jkeating|Jesse Keating]] suggested similar criteria for artwork, spin-kickstarts and fedora-release packages<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094799.html</ref>. Adam provided revised drafts for all three proposed criteria<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094802.html</ref>. | ||
<references/> | |||
=== Testing new versions of anaconda === | |||
[[User: | [[User:mcloaked|Mike Cloaked]] provided a very useful guide to creating a custom USB key to test new releases of anaconda when images containing that version of anaconda are not available<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094752.html</ref>. [[User:Bcl|Brian Lane]] said it was good to see someone else using his work, and pointed out that the script Mike used should also be able to produce a full, updated DVD ISO, but would take longer to do so. | ||
<references/> | <references/> | ||
=== | === Fedora QA retrospective === | ||
[[User: | [[User:Jlaska|James Laska]] announced the QA retrospective page for Fedora 14<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094587.html</ref>. The retrospective attempts to identify things that went well and things that went badly during the release cycle, to help the group improve with future releases. He appealed for contributions to the retrospective from anyone who could identify good or bad elements of the QA work for the Fedora 14 release. | ||
<references/> | <references/> | ||
Revision as of 04:31, 28 October 2010
QualityAssurance
In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].
Contributing Writer: Adam Williamson
Test Days
The last Fedora 14 Test Day[1] on 2010-10-14 was on the use of OpenLDAP with the NSS security library - the use of NSS with OpenLDAP is new in Fedora 14, replacing the previous use of OpenSSL. Kamil Paral provided a recap to the list[2]. He noted that "the participation was little low, but it was somehow expected, because this was a non-trivial topic" and that the event discovered three bugs and confirmed the rest of the functionality worked.
If you would like to propose a main track Test Day for the Fedora 15 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[3].
Fedora 14 Final testing
The QA group put together a great team effort to perform comprehensive and efficient validation testing on the Fedora 14 final release. The Test Compose was announced on 2010-10-13[1], and the installation[2] and desktop[3] test matrices were completed quickly, with results coming in from many different group members. The testing identified several blocker bugs, which were all resolved in time for the release of the Release Candidate on 2010-10-21[4]. Once again, the group came together to perform the installation[5] and desktop[6] validation testing, which was mostly completed by the weekend. No further blocker bugs were discovered (as outlined by Rui He in the RC1 testing recap[7]), and so the QA group was able to join with release engineering and development to sign off on the release of RC1 as Fedora 14 Final at the go/no-go meeting of 2010-10-26[8]. Adam Williamson thanked all of the large number of community members who contributed to the validation testing in a blog post[9].
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-October/094596.html
- ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_TC1_Install
- ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_TC1_Desktop
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-October/094893.html
- ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_RC1_Install
- ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_RC1_Desktop
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-October/094998.html
- ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-10-26/fedora-meeting.2010-10-26-20.59.log.html
- ↑ http://www.happyassassin.net/2010/10/26/fedora-14-goes-gold/
Reporting bugs from downstream distributions
Edward Kirk reported that the lead developer of Fusion Linux, a distribution based on Fedora, was suggesting users file bugs in Fedora Bugzilla[1]. A discussion ensued on whether it was correct for users of distributions downstream of Fedora to report bugs directly to Fedora's bug tracker. Jóhann Guðmundsson felt strongly that it was not appropriate, and such distributions should have their own bug trackers[2]. Michael Schwendt pointed out that Fedora's abrt does not currently make it easy for downstream distributions to modify it to report crashes to a different bug tracking system[3]. Adam Williamson thought it made sense for bugs in Fedora packages which are present unchanged in Fusion Linux to be reported to Fedora's bug tracker, in the same way Fedora bug reports are often sent upstream to GNOME or KDE[4]. The Fusion Linux developer, Valent Turkovic, said that his plan was for Fusion Linux developers to check reports of bugs and ask the user to file them in Fedora Bugzilla if the package was unchanged from Fedora, RPM Fusion Bugzilla if the package came from there, or with Fusion Linux's developers if the bug related to Fusion Linux-specific customizations.
Release criteria
Adam Williamson proposed a new release criterion[1] requiring the final release notes from the Documentation team be present in packaged form in the release repository (at Final release stage). Jesse Keating suggested similar criteria for artwork, spin-kickstarts and fedora-release packages[2]. Adam provided revised drafts for all three proposed criteria[3].
Testing new versions of anaconda
Mike Cloaked provided a very useful guide to creating a custom USB key to test new releases of anaconda when images containing that version of anaconda are not available[1]. Brian Lane said it was good to see someone else using his work, and pointed out that the script Mike used should also be able to produce a full, updated DVD ISO, but would take longer to do so.
Fedora QA retrospective
James Laska announced the QA retrospective page for Fedora 14[1]. The retrospective attempts to identify things that went well and things that went badly during the release cycle, to help the group improve with future releases. He appealed for contributions to the retrospective from anyone who could identify good or bad elements of the QA work for the Fedora 14 release.