(fix a link) |
(create fwn 268 qa beat) |
||
| Line 10: | Line 10: | ||
=== Test Days === | === Test Days === | ||
Thursday 2011-03- | Thursday 2011-03-17 was preupgrade Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-17_Preupgrade</ref>. A good group of testers showed up, but testing was somewhat complicated by underlying bugs in anaconda. However, the testers were able to check that most of preupgrade itself was working as intended. [[User:Rhe|Rui He]] posted a recap to the list<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097977.html</ref>. | ||
This Thursday, 2011-03-24, is power management Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-24</ref>, where we'll be testing how well various common power management operations work on a range of systems. The main focus of this event is to check common power management features on a wide range of hardware, so we really need as many people as possible to come along. Please come out if you have a bit of spare time. | |||
Next | Next Tuesday (note the special day!), 2011-03-29, will be printing Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-29</ref>. We'll be aiming to test out Fedora 15's printing stack with as many different printers as possible, so if you have a printer and a pulse, please come and join us! | ||
Next Thursday, 2011-03-31, will be ABRT Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2011-03-31_ABRT_Retrace_Server</ref>. As well as checking that ABRT (Fedora's automated crash report tool) is working as expected for Fedora 15, we'll be testing out a big new feature, the retrace server<ref>http://fedoraproject.org/wiki/Features/RetraceServer</ref>. This allows you to submit crash reports to a remote server which will generate the backtrace - avoiding the need for you to download and install often large debuginfo packages in order to submit reports. Please come along and help us test this exciting new feature! | |||
<references/> | |||
=== Release criteria revisions === | |||
[[User:Adamwill|Adam Williamson]] proposed several alterations to the release criteria<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097859.html</ref>, covering several issues that had become clear at recent blocker meetings. The alterations attempted to account for bugs that were specific to particular keymaps or languages, cover firstboot, cover unintended installations, and clarify a few other points. Andre Robatino proposed an additional criterion requiring a working media check process in the installer<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097861.html</ref>, an idea Adam supported<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097863.html</ref>. [[User:Jlaska|James Laska]] recommended caution to avoid tying the criterion too closely to a specific implementation<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097870.html</ref>. | |||
<references/> | |||
=== Nightly builds via Koji === | |||
In response to a query from Gregory Woodbury about nightly builds being stalled<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097856.html</ref>, [[User:Kevin|Kevin Fenzi]] explained<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097860.html</ref> that this was due to a problem on the compose machine. He went on to announce that the nightly builds would soon be done via Koji, which will result in the builds happening much faster, guarantee that they are built in exactly the same way as official composes, and allow for an archive of recent nightly builds to be kept, making regression testing easier. | |||
<references/> | <references/> | ||
=== | === QA group governance === | ||
[[User:Johannbg|Jóhann Guðmundsson]] raised some questions regarding the governance of the QA group<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097891.html</ref>, including the idea of an elected board to handle some decision making when representing the group as a whole. [[StephenSmoogen|Stephen Smoogen]] noted that SIGs do not generally have formal voting structures<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097899.html</ref>, while leaving open the question of whether QA is (or should be) a SIG or a Steering Committee. Clyde E. Kunkel noted that the functions of the QA team are mostly advisory rather than decision-making, and expressed a desire for an organization chart (or 'wiring diagram') for the various groups that make up the Fedora project<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097902.html</ref>. [[User:Adamwill|Adam Williamson]] pointed out that the Board had worked on such a diagram at FUDCon Tempe, and a copy of the image may be available<ref>http://lists.fedoraproject.org/pipermail/test/2011-March/097910.html</ref>. He replied to Jóhann's question about the status of Red Hat employees in the group by saying they tried not to see their position as giving them automatic decision-making power on behalf of the group, but admitted that in practice, other groups often seem to come directly to the Red Hat employees in the group and consider their responses as representing the QA group as a whole. | |||
<references/> | <references/> | ||
| Line 26: | Line 40: | ||
=== Fedora 15 Beta preparation === | === Fedora 15 Beta preparation === | ||
The | The second Beta blocker/nice-to-have review meeting took place on 2011-03-18<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2011-03-18/f-15-beta-blocker.2011-03-18-17.00.html</ref>, and the team worked through the full list of proposed Beta blocker and nice-to-have bugs. The first automated Beta acceptance test run was scheduled for 2011-03-17, but this was not possible due to underlying issues in anaconda at the time, as discussed at the weekly meeting of 2011-03-21<ref>http://fedoraproject.org/wiki/QA/Meetings/20110321</ref>. At the same meeting, [[User:Jlaska|James Laska]] noted that the Beta test compose was at risk due to the same problems, and committed to providing updated images to test candidate fixes for the problems. | ||
<references/> | <references/> | ||
Revision as of 03:33, 24 March 2011
QualityAssurance
In this section, we cover the activities of the QA team[1]. For more information on the work of the QA team and how you can get involved, see the Joining page[2].
Contributing Writer: Adam Williamson
Test Days
Thursday 2011-03-17 was preupgrade Test Day[1]. A good group of testers showed up, but testing was somewhat complicated by underlying bugs in anaconda. However, the testers were able to check that most of preupgrade itself was working as intended. Rui He posted a recap to the list[2].
This Thursday, 2011-03-24, is power management Test Day[3], where we'll be testing how well various common power management operations work on a range of systems. The main focus of this event is to check common power management features on a wide range of hardware, so we really need as many people as possible to come along. Please come out if you have a bit of spare time.
Next Tuesday (note the special day!), 2011-03-29, will be printing Test Day[4]. We'll be aiming to test out Fedora 15's printing stack with as many different printers as possible, so if you have a printer and a pulse, please come and join us!
Next Thursday, 2011-03-31, will be ABRT Test Day[5]. As well as checking that ABRT (Fedora's automated crash report tool) is working as expected for Fedora 15, we'll be testing out a big new feature, the retrace server[6]. This allows you to submit crash reports to a remote server which will generate the backtrace - avoiding the need for you to download and install often large debuginfo packages in order to submit reports. Please come along and help us test this exciting new feature!
- ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-17_Preupgrade
- ↑ http://lists.fedoraproject.org/pipermail/test/2011-March/097977.html
- ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-24
- ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-29
- ↑ http://fedoraproject.org/wiki/Test_Day:2011-03-31_ABRT_Retrace_Server
- ↑ http://fedoraproject.org/wiki/Features/RetraceServer
Release criteria revisions
Adam Williamson proposed several alterations to the release criteria[1], covering several issues that had become clear at recent blocker meetings. The alterations attempted to account for bugs that were specific to particular keymaps or languages, cover firstboot, cover unintended installations, and clarify a few other points. Andre Robatino proposed an additional criterion requiring a working media check process in the installer[2], an idea Adam supported[3]. James Laska recommended caution to avoid tying the criterion too closely to a specific implementation[4].
Nightly builds via Koji
In response to a query from Gregory Woodbury about nightly builds being stalled[1], Kevin Fenzi explained[2] that this was due to a problem on the compose machine. He went on to announce that the nightly builds would soon be done via Koji, which will result in the builds happening much faster, guarantee that they are built in exactly the same way as official composes, and allow for an archive of recent nightly builds to be kept, making regression testing easier.
QA group governance
Jóhann Guðmundsson raised some questions regarding the governance of the QA group[1], including the idea of an elected board to handle some decision making when representing the group as a whole. Stephen Smoogen noted that SIGs do not generally have formal voting structures[2], while leaving open the question of whether QA is (or should be) a SIG or a Steering Committee. Clyde E. Kunkel noted that the functions of the QA team are mostly advisory rather than decision-making, and expressed a desire for an organization chart (or 'wiring diagram') for the various groups that make up the Fedora project[3]. Adam Williamson pointed out that the Board had worked on such a diagram at FUDCon Tempe, and a copy of the image may be available[4]. He replied to Jóhann's question about the status of Red Hat employees in the group by saying they tried not to see their position as giving them automatic decision-making power on behalf of the group, but admitted that in practice, other groups often seem to come directly to the Red Hat employees in the group and consider their responses as representing the QA group as a whole.
Fedora 15 Beta preparation
The second Beta blocker/nice-to-have review meeting took place on 2011-03-18[1], and the team worked through the full list of proposed Beta blocker and nice-to-have bugs. The first automated Beta acceptance test run was scheduled for 2011-03-17, but this was not possible due to underlying issues in anaconda at the time, as discussed at the weekly meeting of 2011-03-21[2]. At the same meeting, James Laska noted that the Beta test compose was at risk due to the same problems, and committed to providing updated images to test candidate fixes for the problems.