FWN/Beats/QualityAssurance: Difference between revisions

Revision as of 04:15, 4 November 2010

QualityAssurance

In this section, we cover the activities of the QA team^[1]. For more information on the work of the QA team and how you can get involved, see the Joining page^[2].

Contributing Writer: Adam Williamson

Test Days

The Fedora 14 Test Day cycle has concluded. If you would like to propose a main track Test Day for the Fedora 15 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac^[1].

↑ http://fedorahosted.org/fedora-qa/

Installing Rawhide

Qiang Li asked what was now the recommended method of installing Rawhide^[1], given that a Rawhide installer build is now not always available. Adam Williamson recommended updating from the latest pre-release using yum^[2], to which Qiang replied that he does not like doing this due to the time and bandwidth involved^[3]. Later in the discussion, Christoph Frieben recommended using the latest pre-release installer and specifying Rawhide repositories during the repository selection step^[4]. Rui He^[5] and Jesse Keating^[6] also suggested this method.

Testing updates just prior to release

Kamil Paral asked how one can test the installation of updates shortly before a release, when none are available in the official update repositories^[1]. James Laska recommended downgrading an installed package such as gcalctool^[2], and noted he has a repository available for this purpose.

Fedora 12 critical path testing

Adam Williamson noted a Fedora 12 updates-testing report which listed many critical path updates which had been awaiting the required proventester testing for some weeks^[1]. He proposed removing the proven tester requirement for Fedora 12 critical path updates as a practical measure to allow the updates to go through before EOL. In the mean time, he reported that he had tested several of the updates in a virtual machine^[2], and Gene did the same^[3], allowing several of the updates to go through.

[1] ttp://fedoraproject.org/wiki/QA

[2] ttp://fedoraproject.org/wiki/QA/Join

[3] ttp://fedorahosted.org/fedora-qa/

[4] ttp://lists.fedoraproject.org/pipermail/test/2010-October/094970.html

[5] ttp://lists.fedoraproject.org/pipermail/test/2010-October/094971.html

[6] ttp://lists.fedoraproject.org/pipermail/test/2010-October/094973.html

[7] ttp://lists.fedoraproject.org/pipermail/test/2010-October/094977.html

[8] ttp://lists.fedoraproject.org/pipermail/test/2010-October/094995.html

[9] ttp://lists.fedoraproject.org/pipermail/test/2010-October/094983.html

[10] ttp://lists.fedoraproject.org/pipermail/test/2010-October/095001.html

[11] ttp://lists.fedoraproject.org/pipermail/test/2010-October/095003.html

[12] ttp://lists.fedoraproject.org/pipermail/test/2010-October/095135.html

[13] ttp://lists.fedoraproject.org/pipermail/test/2010-October/095147.html

[14] ttp://lists.fedoraproject.org/pipermail/test/2010-October/095150.html

[1]

[2]

[1]

[1]

[2]

[3]

[4]

[5]

[6]

[1]

[2]

[1]

[2]

[3]

@@ Line 10: / Line 10: @@
 === Test Days ===
-The last Fedora 14 Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-10-14_OpenLDAP/NSS</ref> on 2010-10-14 was on the use of OpenLDAP with the NSS security library - the use of NSS with OpenLDAP is new in Fedora 14, replacing the previous use of OpenSSL. [[User:Kparal|Kamil Paral]] provided a recap to the list<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094698.html</ref>. He noted that "the participation was little low, but it was somehow expected, because this was a non-trivial topic" and that the event discovered three bugs and confirmed the rest of the functionality worked.
+The Fedora 14 Test Day cycle has concluded. If you would like to propose a main track Test Day for the Fedora 15 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
-If you would like to propose a main track Test Day for the Fedora 15 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>.
-<references/>
-=== Fedora 14 Final testing ===
-The QA group put together a great team effort to perform comprehensive and efficient validation testing on the Fedora 14 final release. The Test Compose was announced on 2010-10-13<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094596.html</ref>, and the installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_TC1_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_TC1_Desktop</ref> test matrices were completed quickly, with results coming in from many different group members. The testing identified several blocker bugs, which were all resolved in time for the release of the Release Candidate on 2010-10-21<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094893.html</ref>. Once again, the group came together to perform the installation<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_RC1_Install</ref> and desktop<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_14_Final_RC1_Desktop</ref> validation testing, which was mostly completed by the weekend. No further blocker bugs were discovered (as outlined by [[User:Rhe|Rui He]] in the RC1 testing recap<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094998.html</ref>), and so the QA group was able to join with release engineering and development to sign off on the release of RC1 as Fedora 14 Final at the go/no-go meeting of 2010-10-26<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-10-26/fedora-meeting.2010-10-26-20.59.log.html</ref>. [[User:Adamwill|Adam Williamson]] thanked all of the large number of community members who contributed to the validation testing in a blog post<ref>http://www.happyassassin.net/2010/10/26/fedora-14-goes-gold/</ref>.
-<references/>
-=== Reporting bugs from downstream distributions ===
-[[User:Tk009|Edward Kirk]] reported that the lead developer of Fusion Linux, a distribution based on Fedora, was suggesting users file bugs in Fedora Bugzilla<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094830.html</ref>. A discussion ensued on whether it was correct for users of distributions downstream of Fedora to report bugs directly to Fedora's bug tracker. [[User:Johannbg|Jóhann Guðmundsson]] felt strongly that it was not appropriate, and such distributions should have their own bug trackers<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094844.html</ref>. [[User:Mschwendt|Michael Schwendt]] pointed out that Fedora's abrt does not currently make it easy for downstream distributions to modify it to report crashes to a different bug tracking system<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094848.html</ref>. [[User:Adamwill|Adam Williamson]] thought it made sense for bugs in Fedora packages which are present unchanged in Fusion Linux to be reported to Fedora's bug tracker, in the same way Fedora bug reports are often sent upstream to GNOME or KDE<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094852.html</ref>. The Fusion Linux developer, [[User:Valent|Valent Turkovic]], said that his plan was for Fusion Linux developers to check reports of bugs and ask the user to file them in Fedora Bugzilla if the package was unchanged from Fedora, RPM Fusion Bugzilla if the package came from there, or with Fusion Linux's developers if the bug related to Fusion Linux-specific customizations.
 <references/>
-=== Release criteria ===
+=== Installing Rawhide ===
-[[User:Adamwill|Adam Williamson]] proposed a new release criterion<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094785.html</ref> requiring the final release notes from the Documentation team be present in packaged form in the release repository (at Final release stage). [[User:Jkeating|Jesse Keating]] suggested similar criteria for artwork, spin-kickstarts and fedora-release packages<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094799.html</ref>. Adam provided revised drafts for all three proposed criteria<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094802.html</ref>.
+Qiang Li asked what was now the recommended method of installing Rawhide<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094970.html</ref>, given that a Rawhide installer build is now not always available. [[User:Adamwill|Adam Williamson]] recommended updating from the latest pre-release using yum<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094971.html</ref>, to which Qiang replied that he does not like doing this due to the time and bandwidth involved<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094973.html</ref>. Later in the discussion, Christoph Frieben recommended using the latest pre-release installer and specifying Rawhide repositories during the repository selection step<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094977.html</ref>. [[User:Rhe|Rui He]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094995.html</ref> and [[User:jkeating|Jesse Keating]]<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094983.html</ref> also suggested this method.
 <references/>
-=== Testing new versions of anaconda ===
+=== Testing updates just prior to release ===
-[[User:mcloaked|Mike Cloaked]] provided a very useful guide to creating a custom USB key to test new releases of anaconda when images containing that version of anaconda are not available<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094752.html</ref>. [[User:Bcl|Brian Lane]] said it was good to see someone else using his work, and pointed out that the script Mike used should also be able to produce a full, updated DVD ISO, but would take longer to do so.
+[[User:Kparal|Kamil Paral]] asked how one can test the installation of updates shortly before a release, when none are available in the official update repositories<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/095001.html</ref>. [[User:Jlaska|James Laska]] recommended downgrading an installed package such as gcalctool<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/095003.html</ref>, and noted he has a repository available for this purpose.
 <references/>
-=== Fedora QA retrospective ===
+=== Fedora 12 critical path testing ===
-[[User:Jlaska|James Laska]] announced the QA retrospective page for Fedora 14<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/094587.html</ref>. The retrospective attempts to identify things that went well and things that went badly during the release cycle, to help the group improve with future releases. He appealed for contributions to the retrospective from anyone who could identify good or bad elements of the QA work for the Fedora 14 release.
+[[User:Adamwill|Adam Williamson]] noted a Fedora 12 updates-testing report which listed many critical path updates which had been awaiting the required proventester testing for some weeks<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/095135.html</ref>. He proposed removing the proven tester requirement for Fedora 12 critical path updates as a practical measure to allow the updates to go through before EOL. In the mean time, he reported that he had tested several of the updates in a virtual machine<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/095147.html</ref>, and Gene did the same<ref>http://lists.fedoraproject.org/pipermail/test/2010-October/095150.html</ref>, allowing several of the updates to go through.
 <references/>

Search