FWN/Beats/QualityAssurance: Difference between revisions

Revision as of 05:51, 30 May 2009

QualityAssurance

In this section, we cover the activities of the QA team^[1].

↑ http://fedoraproject.org/wiki/QA

Test Days

There was no Test Day last week, as we are deep in the Fedora 11 final release run-up.

Currently, no Test Day is scheduled for next week - it is too close to the scheduled release of Fedora 11 for any testing to produce results directly in Fedora 11 final release, but if you would like to propose a test day which could result in changes for post-release updates, or an early test day for Fedora 12, please contact the QA team via email or IRC.

Weekly meetings

The QA group weekly meeting^[1] was held on 2009-05-27. The full log is available^[2]. Adam Williamson reported that he had again not yet remembered to ask the Bugzilla team to add a link to the Fedora bug workflow page.

James Laska reported that he was still not yet ready to send out a Test Day feedback survey to previous participants, but continued to work on it.

John Poelstra reported that he had updated the current Fedora 12 schedule^[3].

Will Woods reported that he had added a test case for upgrading from one Fedora release to the next with an encrypted root partition^[4].

The group discussed how to handle the installation test result matrix wiki page^[5] between release candidate revisions. James Laska committed to work out his best solution and send it to the mailing list.

Adam Williamson reported that the cleaning and revising of the Fedora 11 Common Bugs page^[6] was complete. James Laska added that he had as promised been adding significant installation issues to the page. Adam said that he had added the X.org issues of which he was aware, and sound-related issues. He noted that François Cami had created an initial draft of a list of ATI-related issues, but had not yet completed it.

Will Woods clarified that his preferred title in relation to autoqa issues is Cap'n Autoqa. The minutes do not relate whether or not there is a parrot.

The group reviewed the current status of Fedora 11 GA (final release) from its perspective. (Note that this meeting took place before the latest delay in the final ship date). They went over the list of currently open release blocker bugs, and agreed it seemed possible to make the final deadline for initial RC generation with all of the bugs at least tentatively resolved. There was detailed discussion of two bugs (502077 and 498553). Action plans were developed for both issues to have them addressed within the one-day deadline the team was at this point working with.

The Bugzappers group weekly meeting^[7] was held on 2009-05-26. The full log is available^[8]. John Poelstra reported on progress of the housekeeping changes for Fedora 11's release, and the group agreed that he was doing a fine job and should keep it up.

Adam Williamson reported on the progress of the triage metric system. Significant progress had been made during the week by the author, Brennan Ashton. The system is now fully working on the official Fedora infrastructure hosting server^[9]. It is currently working with a test snapshot of data rather than with the live Bugzilla data, but it should already be theoretically capable of working with the live data. The project will now enter a tidying-up and beta testing phase during which it will be brought up to a state where it can be declared fully usable. This should take two weeks or so. The group noted that the list of triagers was based on the FAS 'triagers' group, which leads back to the existing question of how to rationalize the 'fedorabugs' and 'triagers' groups. Brennan will work with Jon Stanley to address this issue.

Adam Williamson also reported on the progress of the proposal to include setting the priority / severity fields as part of triage. As no feedback opposing the Cepl Method^[10] had been received on the mailing list, the group agreed that it could now go ahead and adopt this as the official method of setting severity at the triage stage. Adam said he would work with the Bugzilla team to restrict access to the priority and severity fields as had been agreed as part of the proposal, and then adjust all the relevant documentation on the Wiki to put the severity policy into place.

The next QA weekly meeting will be held on 2009-06-03 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-02 at 1500 UTC in #fedora-meeting.

Unified Greasemonkey triage script

Matej Cepl announced^[1] that he had released a new revised and unified Greasemonkey script for triagers incorporating all features of all previously released scripts. Edward Kirk thanked him for his work^[2]. Steven Parrish noted^[3] that GreaseMonkey did not yet work unmodified with the current Firefox 3.5 pre-release as found in Fedora 11. Matej suggested^[4] the Nightly Tester Tools extension as an easy way to work around this limitation.

Merging Fedora 11 FAQ into other pages

Christopher Beland revived^[1] the idea of merging the Fedora 11 FAQ^[2], maintained by Rahul Sundaram, into other pages, as most of its content could more appropriately be located in various other places, including the Release Notes, Installation Guide, Common Bugs page and other places. Rahul explained^[3] that he was happy for any content that could be moved to a more appropriate place to be removed from the FAQ page. The documentation team's Susan Lauber contributed some suggestions^[4] on other appropriate places the content could be moved to, and in a later thread she provided^[5] some more useful information on adding information to the Release Notes post-freeze.

Release Candidate testing

James Laska announced^[1] testing for the first release candidate build for Fedora 11 (and, later, for the second^[2]). He asked for installation-related issues to be reported to the Wiki test matrix page^[3]. This led indirectly to questions about where to find the release candidate images (their location is buried within the matrix page in order to try and limit demand for the images) and why release candidate images are not more widely promoted and distributed^[4]. Jesse Keating explained ^[5] that the amounts of data were too great, the available storage and bandwidth resources too small, and the timeframes too tight for release candidate images to be meaningfully distributed for public testing. He did emphasize^[6], however, that the community could contribute useful testing through use of the Rawhide repositories and installer images, which currently are synchronized with the release candidate builds.

[1] ttp://fedoraproject.org/wiki/QA

[2] ttp://fedoraproject.org/wiki/QA/Meetings

[3] ttp://fedoraproject.org/wiki/QA/Meetings/20090527

[4] ttp://poelstra.fedorapeople.org/schedules/f-12/f-12-releng-tasks.html

[5] ttps://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_Encrypted_Root

[6] ttps://fedoraproject.org/wiki/QA:Fedora_11_RC2_Install_Test_Results

[7] ttps://fedoraproject.org/wiki/Common_F11_bugs

[8] ttp://fedoraproject.org/wiki/BugZappers/Meetings

[9] ttp://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-May-26

[10] ttp://publictest14.fedoraproject.org/triageweb/

[11] ttp://fedoraproject.org/wiki/User:Beland/Bugzilla_Legend#Proposal_B:

[12] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01131.html

[13] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01132.html

[14] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01163.html

[15] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01174.html

[16] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01226.html

[17] ttp://fedoraproject.org/wiki/Fedora_11_FAQ

[18] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01228.html

[19] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01246.html

[20] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01328.html

[21] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01272.html

[22] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01333.html

[23] ttps://fedoraproject.org/wiki/QA:Fedora_11_RC2_Install_Test_Results

[24] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01308.html

[25] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01309.html

[26] ttp://www.redhat.com/archives/fedora-test-list/2009-May/msg01286.html

[1]

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[1]

[2]

[3]

[4]

[1]

[2]

[3]

[4]

[5]

[1]

[2]

[3]

[4]

[5]

[6]

@@ Line 18: / Line 18: @@
 === Weekly meetings ===
-The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-05-20. The full log is available<ref>http://fedoraproject.org/wiki/QA/Meetings/20090520</ref>. [[User:Adamwill|Adam Williamson]] reported that he had filed a ticket to have Bodhi use the appropriate resolution for bugs fixed with stable release updates. [[User:Lmacken|Luke Macken]] said he would take care of the ticket. Adam also reported that he had not yet remembered to ask the Bugzilla team to add a link to the Fedora bug workflow page.
+The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-05-27. The full log is available<ref>http://fedoraproject.org/wiki/QA/Meetings/20090527</ref>. [[User:Adamwill|Adam Williamson]] reported that he had again not yet remembered to ask the Bugzilla team to add a link to the Fedora bug workflow page.
-[[User:Jlaska|James Laska]] reported that he had not yet sent out a Test Day feedback survey to previous participants, but had a draft ready and would continue to work on it.
+[[User:Jlaska|James Laska]] reported that he was still not yet ready to send out a Test Day feedback survey to previous participants, but continued to work on it.
-[[User:poelstra|John Poelstra]] said that he had not yet finalized a schedule for blocker bug reviews during the Fedora 12 cycle, as the overall Fedora 12 cycle was still not finalized. He will revisit the issue next week.
+[[User:poelstra|John Poelstra]] reported that he had updated the current Fedora 12 schedule<ref>http://poelstra.fedorapeople.org/schedules/f-12/f-12-releng-tasks.html</ref>.
-[[User:Wwoods|Will Woods]] reported there had been little work on the autoqa project or adding upgrade test cases to the Wiki during the week, as testing for Fedora 11 release had taken priority.
+[[User:Wwoods|Will Woods]] reported that he had added a test case for upgrading from one Fedora release to the next with an encrypted root partition<ref>https://fedoraproject.org/wiki/QA:Testcase_Anaconda_Upgrade_Encrypted_Root</ref>.
-The group discussed the Fedora 11 release situation, and noted that the release had been pushed back one week. They examined the blocker bug list, and found it was generally manageable. Francois Cami noted that major fixes to X.org's core or the ATI driver were unlikely as [[DaveAirlie|Dave Airlie]] is on vacation. The group noted that most remaining X.org blockers were in the Intel driver, and assigned to
+The group discussed how to handle the installation test result matrix wiki page<ref>https://fedoraproject.org/wiki/QA:Fedora_11_RC2_Install_Test_Results</ref> between release candidate revisions. [[User:Jlaska|James Laska]] committed to work out his best solution and send it to the mailing list.
-The group then discussed the Fedora 11 Common Bugs page<ref>http://fedoraproject.org/wiki/Common_F11_bugs</ref>. [[User:Adamwill|Adam Williamson]] volunteered to revise the existing page to match the format used for previous releases, initiate a few other changes based on his previous work on Mandriva Linux Errata pages, and talk to other groups about the use of the page, including the Documentation group, and the IRC, mailing list and forum support teams. Francois Cami volunteered to help ensure all appropriate X.org issues are tracked on the page.
+[[User:Adamwill|Adam Williamson]] reported that the cleaning and revising of the Fedora 11 Common Bugs page<ref>https://fedoraproject.org/wiki/Common_F11_bugs</ref> was complete. [[User:Jlaska|James Laska]] added that he had as promised been adding significant installation issues to the page. Adam said that he had added the X.org issues of which he was aware, and sound-related issues. He noted that François Cami had created an initial draft of a list of ATI-related issues, but had not yet completed it.
-In open discussion, Francois Cami noted that for Fedora 11, users of several generations of ATI Radeon cards would now only have the free drivers as a viable choice, whereas previously they would be able to use the ATI proprietary driver from third-party repositories. It was also noted that, at release time, the commonly-used RPMFusion repository would have no packages available for the proprietary driver even for cards for which it is still available, due to the lack of a reliable patch for kernel 2.6.29. The group discussed whether this could be explained in the release notes (with no definite resolution, but advice to check with the Documentation team), and how to note the requirements for a full and useful bug report for problems with the free driver from such users. [[User:Adamwill|Adam Williamson]] noted that the appropriate venue for such information would be the Bugs and Feature Requests Wiki page<ref>http://fedoraproject.org/wiki/BugsAndFeatureRequests</ref>.
+[[User:Wwoods|Will Woods]] clarified that his preferred title in relation to autoqa issues is Cap'n Autoqa. The minutes do not relate whether or not there is a parrot.
-The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-05-19. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-May-19</ref>. [[User:poelstra|John Poelstra]] reported on progress of the housekeeping changes for Fedora 11's release, and the group agreed that he was doing a fine job and should keep it up.
+The group reviewed the current status of Fedora 11 GA (final release) from its perspective. (Note that this meeting took place before the latest delay in the final ship date). They went over the list of currently open release blocker bugs, and agreed it seemed possible to make the final deadline for initial RC generation with all of the bugs at least tentatively resolved. There was detailed discussion of two bugs (502077 and 498553). Action plans were developed for both issues to have them addressed within the one-day deadline the team was at this point working with.
-[[User:Adamwill|Adam Williamson]] reported on the progress of the triage metric system. Brennan Ashton has again been busy (and without access to a regular internet connection), so progress has been slow. Adam clarified that the main choke point now was the lack of a set of test data for the scripts, to ensure that they were working correctly, and explained he was doing his best to get this data made available. Once it is available, work on the system is no longer solely dependent on Brennan being available.
+The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-05-26. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-May-26</ref>. [[User:poelstra|John Poelstra]] reported on progress of the housekeeping changes for Fedora 11's release, and the group agreed that he was doing a fine job and should keep it up.
-[[User:Adamwill|Adam Williamson]] also reported on the progress of the proposal to include setting the priority / severity fields as part of triage. He had sent out the mail asking for feedback on how to proceed, but had received none yet. The group agreed that he should send out another mail as a new thread, and set a deadline of the end of the week; if no significant feedback to the contrary was received by that point, the group agreed they should proceed using the method proposed by Matej Cepl.
+[[User:Adamwill|Adam Williamson]] reported on the progress of the triage metric system. Significant progress had been made during the week by the author, Brennan Ashton. The system is now fully working on the official Fedora infrastructure hosting server<ref>http://publictest14.fedoraproject.org/triageweb/</ref>. It is currently working with a test snapshot of data rather than with the live Bugzilla data, but it should already be theoretically capable of working with the live data. The project will now enter a tidying-up and beta testing phase during which it will be brought up to a state where it can be declared fully usable. This should take two weeks or so. The group noted that the list of triagers was based on the FAS 'triagers' group, which leads back to the existing question of how to rationalize the 'fedorabugs' and 'triagers' groups. Brennan will work with [[User:Jstanley|Jon Stanley]] to address this issue.
-The next QA weekly meeting will be held on 2009-05-27 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-05-26 at 1500 UTC in #fedora-meeting.
+[[User:Adamwill|Adam Williamson]] also reported on the progress of the proposal to include setting the priority / severity fields as part of triage. As no feedback opposing the Cepl Method<ref>http://fedoraproject.org/wiki/User:Beland/Bugzilla_Legend#Proposal_B:</ref> had been received on the mailing list, the group agreed that it could now go ahead and adopt this as the official method of setting severity at the triage stage. Adam said he would work with the Bugzilla team to restrict access to the priority and severity fields as had been agreed as part of the proposal, and then adjust all the relevant documentation on the Wiki to put the severity policy into place.
-<references/>
+The next QA weekly meeting will be held on 2009-06-03 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-06-02 at 1500 UTC in #fedora-meeting.
-=== Blocker bug review meeting ===
-[[User:poelstra|John Poelstra]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg00946.html</ref> a joint QA / Release Engineering Fedora 11 blocker bug review meeting on Friday 2009-05-22. The meeting was not logged, but all outstanding release blockers were reviewed, some were closed or downgraded, and action plans were decided for several.
-<references/>
-=== Adobe Flash installation instructions ===
-[[User:Beland|Christopher Beland]] explained<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01004.html</ref> that he had updated the Wiki page on Flash<ref>http://fedoraproject.org/wiki/Flash</ref> with the latest instructions on installing it on x86-64 systems. [[User:Adamwill|Adam Williamson]] noted<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01006.html</ref> that the page should emphasize free software alternatives as well as the proprietary Adobe Flash system. [[User:Pfrields | Paul Frields]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01027.html</ref> that he had made such a change.
 <references/>
-=== Fedora 11 Common Bugs page ===
+=== Unified Greasemonkey triage script ===
-[[User:Adamwill|Adam Williamson]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01026.html</ref> his revisions to the Fedora 11 Common Bugs page, and asked the group to contribute to expanding and maintaining the page and consistently refer to it when explaining problems. He also explained some of the planned improvements to the content of the page and its interaction with Bugzilla.
+Matej Cepl announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01131.html</ref> that he had released a new revised and unified Greasemonkey script for triagers incorporating all features of all previously released scripts. [[User:Tk009|Edward Kirk]] thanked him for his work<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01132.html</ref>. [[User:StevenParrish | Steven Parrish]] noted<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01163.html</ref> that GreaseMonkey did not yet work unmodified with the current Firefox 3.5 pre-release as found in Fedora 11. Matej suggested<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01174.html</ref> the Nightly Tester Tools extension as an easy way to work around this limitation.
 <references/>
-=== Mozilla / Beagle blocker bug proposal ===
+=== Merging Fedora 11 FAQ into other pages ===
-Jonathan Kamens asked<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01041.html</ref> whether a bug preventing Beagle from searching Mozilla Firefox and Thunderbird data should be a release blocker, on the basis that desktop search is a key function for some users and web browser and email data are important sets which someone may wish to search. [[User:bpepple| Brian Pepple]] pointed out<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01042.html</ref> that it did not meet the official release blocker criteria<ref>http://fedoraproject.org/wiki/QA/ReleaseCriteria</ref>, but Jonathan responded that the page admits this is a subjective judgment, and many bugs that do not strictly meet those criteria are in fact considered blockers. In the end, it was mostly agreed that, because Beagle is not installed by default and so is not considered core functionality, the issue should not be considered a blocker.
+[[User:Beland|Christopher Beland]] revived<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01226.html</ref> the idea of merging the Fedora 11 FAQ<ref>http://fedoraproject.org/wiki/Fedora_11_FAQ</ref>, maintained by [[User:Sundaram | Rahul Sundaram]], into other pages, as most of its content could more appropriately be located in various other places, including the Release Notes, Installation Guide, Common Bugs page and other places. Rahul explained<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01228.html</ref> that he was happy for any content that could be moved to a more appropriate place to be removed from the FAQ page. The documentation team's [[User:Laubersm|Susan Lauber]] contributed some suggestions<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01246.html</ref> on other appropriate places the content could be moved to, and in a later thread she provided<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01328.html</ref> some more useful information on adding information to the Release Notes post-freeze.
 <references/>
-=== New Bugzappers ===
+=== Release Candidate testing ===
-Two new Bugzappers volunteers introduced themselves this week: Xia Shing Zee<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg00734.html</ref> and Alex Turner<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01028.html</ref>.
+[[User:Jlaska|James Laska]] announced<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01272.html</ref> testing for the first release candidate build for Fedora 11 (and, later, for the second<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01333.html</ref>). He asked for installation-related issues to be reported to the Wiki test matrix page<ref>https://fedoraproject.org/wiki/QA:Fedora_11_RC2_Install_Test_Results</ref>. This led indirectly to questions about where to find the release candidate images (their location is buried within the matrix page in order to try and limit demand for the images) and why release candidate images are not more widely promoted and distributed<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01308.html</ref>. [[User:jkeating|Jesse Keating]] explained <ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01309.html</ref> that the amounts of data were too great, the available storage and bandwidth resources too small, and the timeframes too tight for release candidate images to be meaningfully distributed for public testing. He did emphasize<ref>http://www.redhat.com/archives/fedora-test-list/2009-May/msg01286.html</ref>, however, that the community could contribute useful testing through use of the Rawhide repositories and installer images, which currently are synchronized with the release candidate builds.
 <references/>

Search