Introduction
This page is intended to gather feedback from the Fedora QA community on things that worked well and things that could have been better with the testing of Fedora 21. The feedback will be used as a basis for identifying areas for improvement for Fedora 22 testing. Any thoughts, big or small, are valuable. If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.
For any questions or concerns, send mail to test@lists.fedoraproject.org.
Providing feedback
- Gwjasu - I like ____ about the new ____ process
Adding feedback is fairly straight forward. If you already have a Fedora account ...
- Login to the wiki
- Select [Edit] for the appropriate section below.
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Anonymous
If you do not have a Fedora account, follow the instructions below to submit anonymous feedback. Please note, mediawiki records the IP address associated with any anonymous page edits.
- Select the appropriate page for your feedback...
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Feedback
Things that went well
- Kparal (talk) - The Rawhide monthly validation pages were a great idea, I think. We captured some problems before Branching, which is always helpful.
- Kparal (talk) - This cycle I tried to convince people to complete all test cases at least once in every milestone, i.e. test even Final test cases during Alpha and Beta, unless those test cases were blocked by something else. I think that was quite successful, we didn't have that many late surprises as in previous cycles. So I consider that as a success. However, I think this approach could still use some improvement. We should make it our official goal, and maybe even somehow track it - we can use the validation statistics for it, but there's nothing reminding us to do it.
- mccann2 - As a newbie tester, I found the wiki test cases well documented and easy to follow. Also found benefit from that 'other page' that showed test coverage to know what was lacking in tests.
Could have been better
- Kparal (talk) - We should revive our approach of contacting individual developers before a blocker bug meeting and invite them there. Of course this consumes more of our time. But I think it really helps (and speeds up) the blocker bug meeting, on the other hand. There were some proposals from Anaconda devs for e.g. sorting the bugs discussed by package maintainer (person or team), not by package name in alphabetical order. That way we can invite a person and discuss all of his/her packages together. We could also compute some statistics about the average time a bug gets discussed, and provide a rough idea of time when the bug is going to get discussed. We should point out that these people don't need to attend the meeting, just lurk in the channel and be available when we ping them, roughly about that time. If they are not available to come, they should provide some information in the bug itself. We can create templates describing all of this, and paste them into bugzilla + send them to the person's email and irc.
- Bruno A late change to the XFCE spin that increased its size by about 200 MB and affected the Games Spin (which is based on XFCE), but not until the first final RC. It might have too late to get the Games spin under its targeted size if that RC had been accepted for release. The XFCE package changed size because the target for its size was changed and more packages were added (well unsubtracted) for it in its kickstart file. This wasn't picked up right away, but when another change that was added to spin-kickstarts was needed, the XFCE change was picked up as well.
Wishlist
- Kparal (talk) - I wish there was a different process than the blocker bug process for product working groups' requirements, like desktop icons polish. It doesn't really make sense to block all products indefinitely just because of some polish needs. I think a separate process with clearly defined restrictions could be better. For example, "every product working group can delay each milestone release for up to one week because of their internal product requirements, provided that the milestone is not already 4 or more weeks behind of schedule, and the WG has not been aware of the problem in question for more than 2 weeks". Something like that. Serious thinking needed.
Recommendations
After enough time has been given for feedback, the QA team will discuss and make recommendations on changes to prioritize for Fedora 22. This section organizes and lists the recommendations.
In order to coordinate efforts, and measure effectiveness of recommendations, please record and track any action taken in the Fedora 22 roadmap in the QA TRAC instance.