QualityAssurance
In this section, we cover the activities of the QA team[1].
Contributing Writer: Adam Williamson
Test Days
This week's Test Day[1] was on UEFI[2], the modern BIOS replacement technology which sees increased support in Fedora 11. which has seen substantial changes for Fedora 11. This was mostly an 'internal' test day, as hardware support UEFI is not yet widely available.
Next week is currently planned to have two Test Days. The first will be a follow up to the earlier Anaconda storage rewrite Test Day[3], to check how progress on Anaconda's storage code rewrite is going and whether the significant issues reported at the earlier Test Day have been addressed. The second will be on the Presto plugin for yum[4], which adds support for deltarpms - incrementally diffed packages. This is planned to be a new feature of Fedora 11 if it works well enough, so please come out to help test it. This feature is of particular interest to those with slower internet connections, or bandwidth restrictions.
Weekly meetings
The QA group weekly meeting[1] was held on 2009-04-08. The full log is available[2]Jesse Keating reported good progress with the autoqa project: a post-tree-compose monitor has been written, meaning autoqa is now run automatically when a new Rawhide tree is created. James Laska mentioned that autoqa results are currently being sent to a mailing list[3]. Will Woods mapped out some potential future improvements in the reporting of results. Jesse noted that a major next step will be doing automatic tests following the builds of packages through Koji, to give immediate feedback on issues to maintainers.
Adam Williamson reported that, as requested at last week's meeting, he had filed a bug on the issue with creating live USB images for Fedora 11 trees from Fedora 10. It had been closed as a duplicate, and the original bug subsequently marked as closed in Rawhide. The group concurred that it didn't make sense to say an issue that mostly affected Fedora 10 was fixed in Rawhide, so Adam has posted a further comment to the bug asking for clarification.
James Laska said he had checked in with Warren Togami about the new blog[4] he had created to monitor issues in Rawhide. Warren said that it is intended to be a low traffic blog that includes imformation on non-obvious (potentially high-impact) problems affecting rawhide users. Submissions can be made to Warren for posting on the blog.
James Laska reported that he has not yet been able to discuss the integration of lab-in-a-box and autoqa with Will Woods. He clarified that the idea is to make it possible to do fully automated installation, testing and reporting of test results on Rawhide within a virtualized guest system.
The group discussed the state of play with regards to Fedora 11's final release, particularly whether all appropriate bugs are marked as F11Blocker or F11Target. Members of the group are encouraged to help make sure that all sufficiently important issues are marked as blocking one or the other, and non-serious issues are taken off the F11Blocker list so development resources can be directed appropriately to the most important issues. Everyone agreed that a special meeting should be held soon to work on these lists, and it would be best to involve as many members of the QA and Bugzappers communities as possible.
James Laska committed to working on a simple system for customization of Test Day Live CD images with useful links and information for the particular Test Day for which they are built.
The Bugzappers group weekly meeting[5] was held on 2009-04-07. The full log is available[6]. The group thanked Christopher Beland for his excellent work on the new How to Triage page draft[7], and discussed further revisions to it. It was agreed that checking for upstream reports when triaging bugs should not be mandatory, but recommended. There was a long discussion on whether triagers should be required to follow up on bugs they set to NEEDINFO state, or whether a separate group of triagers should be responsible for following up NEEDINFO bugs. Eventually it was agreed that front-line triagers should be encouraged to follow up themselves, but a regular sweep could also be made by other Bugzappers to catch bugs in NEEDINFO state that were not being followed up. No agreement was made on whether bugs in NEEDINFO state should be closed after 30 days of inactivity, or 60.
Brennan Ashton (comphappy) reported that his Bugzappers metric reporting system is in the process of moving to the Fedora infrastructure, and a beta site should be available in the next few days. He expects the site to be usable by next week, and requests feedback at that time. The group agreed to aim for a 'production' release by May 6th.
Adam Williamson reported that he is drafting an email to devel-list to solicit developer input on the use of the priority and severity fields in Bugzilla, and he will send this draft to test-list for the group's approval before sending it to devel-list.
The next QA weekly meeting will be held on 2009-04-15 at 1600 UTC in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-04-14 at 1500 UTC in #fedora-meeting.
- ↑ http://fedoraproject.org/wiki/QA/Meetings
- ↑ http://www.happyassassin.net/extras/fedora-qa-20090408.log
- ↑ https://fedorahosted.org/pipermail/autoqa-results/2009-April/thread.html
- ↑ http://rawhidewatch.wordpress.com/
- ↑ http://fedoraproject.org/wiki/BugZappers/Meetings
- ↑ http://fedoraproject.org/wiki/BugZappers/Meetings/Minutes-2009-Apr-07
- ↑ https://fedoraproject.org/wiki/User:Beland/How_to_Triage
Triage Day
A successful Triage Day[1] was held on 2009-04-07 following the weekly Bugzappers meeting. Adam Williamson and Edward Kirk helped to train two new triagers, Haase Niels (arxs) and Scott Glaser (Sonar_Guy).
DeviceKit testing
Matthias Clasen notified the QA group[1] that a new version of DeviceKit, which should fix several bugs reported during the previously held Test Day, had landed in Rawhide, and suggested that those who had encountered issues on the Test Day should re-test with the new code.
Triage Days on the Wiki
Adam Williamson apologized for the delay, and announced [1] that a Triage Day page was now available on the Wiki, explaining the existence and function of the Bugzappers group's weekly Triage Day.
Fedora 11 Blocker bugs review
Further to the discussion in the meeting, Adam Williamson requested[1] the group's help in reviewing the list of blocker bugs for Fedora 11 release.
Graphics card Test Day metrics
Adam Williamson explained[1] that he had generated metrics for bug reports from the graphics card Test Days, and posted them to his blog[2].