From Fedora Project Wiki

< FWN‎ | Beats

(create draft for fwn 171)
(creat half-assed 172 qa beat)
Line 10: Line 10:
=== Test Days ===
=== Test Days ===


This week's Test Day<ref>https://fedoraproject.org/wiki/Test_Day:Radeon_2009-04-09</ref> was on UEFI<ref>https://fedoraproject.org/wiki/Features/EFI</ref>, the modern BIOS replacement technology which sees increased support in Fedora 11. which has seen substantial changes for Fedora 11. This was mostly an 'internal' test day, as hardware support UEFI is not yet widely available.
This week saw two Test Days. The first<ref>http://fedoraproject.org/wiki/Test_Day:AnacondaStorageRewrite_2009-04-14</ref> was a follow-up on the Fedora 11 rewrite of Anaconda's storage device code<ref>http://fedoraproject.org/wiki/Features/AnacondaStorageRewrite</ref>. The second<ref>http://fedoraproject.org/wiki/Test_Day:Presto_2009-04-16</ref> was on the Presto plugin for yum, which enables the use of deltarpms for updates. The Anaconda test day verified that many issues from the earlier test day had been resolved and turned up several new bugs, many of which have been fixed already. The Presto test day was surprisingly uneventful: there was good participation but few bugs were discovered, the system worked well and reliably for almost every test.


Next week is currently planned to have two Test Days. The first will be a follow up to the earlier Anaconda storage rewrite Test Day<ref>https://fedoraproject.org/wiki/QA/Test_Days/2009-03-05</ref>, to check how progress on Anaconda's storage code rewrite is going and whether the significant issues reported at the earlier Test Day have been addressed. The second will be on the Presto plugin for yum<ref>https://fedoraproject.org/wiki/Features/Presto</ref>, which adds support for deltarpms - incrementally diffed packages. This is planned to be a new feature of Fedora 11 if it works well enough, so please come out to help test it. This feature is of particular interest to those with slower internet connections, or bandwidth restrictions.
Next week's Test Day<ref>https://fedoraproject.org/wiki/Test_Day:2009-04-21</ref> will be on the minimal platform feature<ref>https://fedoraproject.org/wiki/Features/MinimalPlatform</ref>, support for very small minimal installations. This is another test day which will require installation, so if you are interested in taking part, please make sure to have a spare system or partition on which you can install a Rawhide system. Of course, this week it only needs to be small!


<references/>
<references/>
Line 18: Line 18:
=== Weekly meetings ===
=== Weekly meetings ===


The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-04-08. The full log is available<ref>http://www.happyassassin.net/extras/fedora-qa-20090408.log</ref>[[User:jkeating|Jesse Keating]] reported good progress with the autoqa project: a post-tree-compose monitor has been written, meaning autoqa is now run automatically when a new Rawhide tree is created. [[User:Jlaska|James Laska]] mentioned that autoqa results are currently being sent to a mailing list<ref>https://fedorahosted.org/pipermail/autoqa-results/2009-April/thread.html</ref>. [[User:Wwoods|Will Woods]] mapped out some potential future improvements in the reporting of results. Jesse noted that a major next step will be doing automatic tests following the builds of packages through Koji, to give immediate feedback on issues to maintainers.
The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2009-04-15. The full log is available<ref>http://www.happyassassin.net/extras/fedora-qa-20090415.log</ref>. The group briefly discussed [[User:Jlaska|James Laska's]] plan to improve the customization possibilities for Test Day live CDs. James promised to send a mail to the list regarding his ideas here.


[[User:Adamwill|Adam Williamson]] reported that, as requested at last week's meeting, he had filed a bug on the issue with creating live USB images for Fedora 11 trees from Fedora 10. It had been closed as a duplicate, and the original bug subsequently marked as closed in Rawhide. The group concurred that it didn't make sense to say an issue that mostly affected Fedora 10 was fixed in Rawhide, so Adam has posted a further comment to the bug asking for clarification.
[[User:Adamwill|Adam Williamson]] reported that he had successfully had a post on the Rawhide nss / x86-64 issue added to the rawhidewatch blog<ref>http://rawhidewatch.wordpress.com/</ref>, run by Warren Togami.


[[User:Jlaska|James Laska]] said he had checked in with [[User:Wtogami| Warren Togami]] about the new blog<ref>http://rawhidewatch.wordpress.com/</ref> he had created to monitor issues in Rawhide. Warren said that it is intended to be a low traffic blog that includes imformation on non-obvious (potentially high-impact) problems affecting rawhide users. Submissions can be made to Warren for posting on the blog.
[[User:Adamwill|Adam Williamson]] reported on his progress in evaluating whether important bugs reported in the X driver Test Days are fully repesented on the Fedora 11 release blocker bugs list. The nouveau maintainer, Ben Skeggs, has already reviewed all nouveau bugs. Review of intel and radeon bugs in in process together with the regular triagers for these components, Matej Cepl and Francois Cami.


[[User:Jlaska|James Laska]] reported that he has not yet been able to discuss the integration of lab-in-a-box and autoqa with [[User:Wwoods|Will Woods]]. He clarified that the idea is to make it possible to do fully automated installation, testing and reporting of test results on Rawhide within a virtualized guest system.
[[User:Wwoods|Will Woods]] provided an update on his progress in checking on PulseAudio's readiness for a Fedora 11 release. He noted that some significant problems remained in two ALSA drivers - snd-intel-hda and snd-intel8x0 - which cause problems in PulseAudio. These drivers are used by a very large amount of current sound hardware. However, patches to fix several problematic cases have been added to the Rawhide kernel recently, and the remaining problems can be worked around if fixes are not integrated prior to release time, so it should be possible to release Fedora 11 with a fairly reliable PulseAudio. The group discussed whether it would make sense to schedule a Test Day for Intel audio chipsets, but concluded it was too close to release time and the Test Day schedule was already too busy to make it practical.


The group discussed the state of play with regards to Fedora 11's final release, particularly whether all appropriate bugs are marked as F11Blocker or F11Target. Members of the group are encouraged to help make sure that all sufficiently important issues are marked as blocking one or the other, and non-serious issues are taken off the F11Blocker list so development resources can be directed appropriately to the most important issues. Everyone agreed that a special meeting should be held soon to work on these lists, and it would be best to involve as many members of the QA and Bugzappers communities as possible.
The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-04-14. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Apr-14</ref>. The meeting opened with a call for the Bugzappers group to be proactive in adding serious bugs to the Fedora 11 Blocker and Target bug lists. Several group members expressed the concern that they would not be able accurately to identify which bugs should be added to the list, so [[User:Adamwill|Adam Williamson]] and [[User:Jlaska|James Laska]] promised to discuss the issue at the next QA meeting and see if there was a way to provide firmer policies and guidance in future.


[[User:Jlaska|James Laska]] committed to working on a simple system for customization of Test Day Live CD images with useful links and information for the particular Test Day for which they are built.
The group agreed to delegate the creation and organization of a Wiki area covering SOPs (Standard Operating Procedures) to [[User:poelstra|John Poelstra]].


The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2009-04-07. The full log is available<ref>http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Apr-07</ref>. The group thanked [[User:Beland|Christopher Beland]] for his excellent work on the new How to Triage page draft<ref>https://fedoraproject.org/wiki/User:Beland/How_to_Triage</ref>, and discussed further revisions to it. It was agreed that checking for upstream reports when triaging bugs should not be mandatory, but recommended. There was a long discussion on whether triagers should be required to follow up on bugs they set to NEEDINFO state, or whether a separate group of triagers should be responsible for following up NEEDINFO bugs. Eventually it was agreed that front-line triagers should be encouraged to follow up themselves, but a regular sweep could also be made by other Bugzappers to catch bugs in NEEDINFO state that were not being followed up. No agreement was made on whether bugs in NEEDINFO state should be closed after 30 days of inactivity, or 60.
The discussion about how long to wait before closing NEEDINFO bugs was resolved by a proposal from [[User:poelstra|John Poelstra]]: whether to close after 30 or 60 days will be left to the discretion of individual triagers, while if there is in future any co-ordinated team working to resolve stale NEEDINFO issues not handled by the initial triager, they will use the 60 day method.
 
Brennan Ashton (comphappy) reported that his Bugzappers metric reporting system is in the process of moving to the Fedora infrastructure, and a beta site should be available in the next few days. He expects the site to be usable by next week, and requests feedback at that time. The group agreed to aim for a 'production' release by May 6th.
 
[[User:Adamwill|Adam Williamson]] reported that he is drafting an email to devel-list to solicit developer input on the use of the priority and severity fields in Bugzilla, and he will send this draft to test-list for the group's approval before sending it to devel-list.
 
The next QA weekly meeting will be held on 2009-04-15 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-04-14 at 1500 UTC in #fedora-meeting.
 
<references/>
 
=== Triage Day ===
 
A successful Triage Day<ref>https://fedoraproject.org/wiki/BugZappers/Triage_days</ref> was held on 2009-04-07 following the weekly Bugzappers meeting. [[User:Adamwill|Adam Williamson]] and [[User:Tk009|Edward Kirk]] helped to train two new triagers, Haase Niels (arxs) and Scott Glaser (Sonar_Guy).
 
<references/>
 
=== DeviceKit testing  ===
 
[[MatthiasClasen|Matthias Clasen]] notified the QA group<ref>http://www.redhat.com/archives/fedora-test-list/2009-April/msg00447.html</ref> that a new version of DeviceKit, which should fix several bugs reported during the previously held Test Day, had landed in Rawhide, and suggested that those who had encountered issues on the Test Day should re-test with the new code.
 
<references/>
 
=== Triage Days on the Wiki ===
 
[[User:Adamwill|Adam Williamson]] apologized for the delay, and announced <ref>https://www.redhat.com/archives/fedora-test-list/2009-March/msg01129.html</ref> that a Triage Day page was now available on the Wiki, explaining the existence and function of the Bugzappers group's weekly Triage Day.
 
<references/>
 
=== Fedora 11 Blocker bugs review ===
 
Further to the discussion in the meeting, [[User:Adamwill|Adam Williamson]] requested<ref>http://www.redhat.com/archives/fedora-test-list/2009-April/msg00544.html</ref> the group's help in reviewing the list of blocker bugs for Fedora 11 release.
 
<references/>
 
=== Graphics card Test Day metrics ===
 
[[User:Adamwill|Adam Williamson]] explained<ref>https://www.redhat.com/archives/fedora-test-list/2009-April/msg00549.html</ref> that he had generated metrics for bug reports from the graphics card Test Days, and posted them to his blog<ref>http://www.happyassassin.net/2009/04/08/test-day-metrics/</ref>.


<references/>
<references/>

Revision as of 03:03, 20 April 2009

QualityAssurance

In this section, we cover the activities of the QA team[1].

Contributing Writer: Adam Williamson

Test Days

This week saw two Test Days. The first[1] was a follow-up on the Fedora 11 rewrite of Anaconda's storage device code[2]. The second[3] was on the Presto plugin for yum, which enables the use of deltarpms for updates. The Anaconda test day verified that many issues from the earlier test day had been resolved and turned up several new bugs, many of which have been fixed already. The Presto test day was surprisingly uneventful: there was good participation but few bugs were discovered, the system worked well and reliably for almost every test.

Next week's Test Day[4] will be on the minimal platform feature[5], support for very small minimal installations. This is another test day which will require installation, so if you are interested in taking part, please make sure to have a spare system or partition on which you can install a Rawhide system. Of course, this week it only needs to be small!

Weekly meetings

The QA group weekly meeting[1] was held on 2009-04-15. The full log is available[2]. The group briefly discussed James Laska's plan to improve the customization possibilities for Test Day live CDs. James promised to send a mail to the list regarding his ideas here.

Adam Williamson reported that he had successfully had a post on the Rawhide nss / x86-64 issue added to the rawhidewatch blog[3], run by Warren Togami.

Adam Williamson reported on his progress in evaluating whether important bugs reported in the X driver Test Days are fully repesented on the Fedora 11 release blocker bugs list. The nouveau maintainer, Ben Skeggs, has already reviewed all nouveau bugs. Review of intel and radeon bugs in in process together with the regular triagers for these components, Matej Cepl and Francois Cami.

Will Woods provided an update on his progress in checking on PulseAudio's readiness for a Fedora 11 release. He noted that some significant problems remained in two ALSA drivers - snd-intel-hda and snd-intel8x0 - which cause problems in PulseAudio. These drivers are used by a very large amount of current sound hardware. However, patches to fix several problematic cases have been added to the Rawhide kernel recently, and the remaining problems can be worked around if fixes are not integrated prior to release time, so it should be possible to release Fedora 11 with a fairly reliable PulseAudio. The group discussed whether it would make sense to schedule a Test Day for Intel audio chipsets, but concluded it was too close to release time and the Test Day schedule was already too busy to make it practical.

The Bugzappers group weekly meeting[4] was held on 2009-04-14. The full log is available[5]. The meeting opened with a call for the Bugzappers group to be proactive in adding serious bugs to the Fedora 11 Blocker and Target bug lists. Several group members expressed the concern that they would not be able accurately to identify which bugs should be added to the list, so Adam Williamson and James Laska promised to discuss the issue at the next QA meeting and see if there was a way to provide firmer policies and guidance in future.

The group agreed to delegate the creation and organization of a Wiki area covering SOPs (Standard Operating Procedures) to John Poelstra.

The discussion about how long to wait before closing NEEDINFO bugs was resolved by a proposal from John Poelstra: whether to close after 30 or 60 days will be left to the discretion of individual triagers, while if there is in future any co-ordinated team working to resolve stale NEEDINFO issues not handled by the initial triager, they will use the 60 day method.