| Line 27: | Line 27: | ||
# [[User:jlaska|jlaska]] - ''Beta released on time'' - Without a strong turn-out from the fedora-qa community, there is ''little'' chance that we would had the test results needed in order to properly assert that the beta release criteria had been met. | # [[User:jlaska|jlaska]] - ''Beta released on time'' - Without a strong turn-out from the fedora-qa community, there is ''little'' chance that we would had the test results needed in order to properly assert that the beta release criteria had been met. | ||
# [[User:jlaska|jlaska]] - ''Test Matrix'' - identifying concerns with the Fedora 13 test matrices, and implementing improvements for Fedora 14 worked really well. I really like to revised organization and cleaning of obsolete tests in the installation matrix (see [[QA:Fedora_14_Install_Results_Template]]). | |||
=== Could have been better === | === Could have been better === | ||
Revision as of 19:35, 12 October 2010
Introduction
This page is intended to gather feedback from the Fedora QA community on things that worked well and things that could have been better with the testing of Fedora 14. The feedback will be used as a basis for identifying areas for improvement for Fedora 14 testing. Any thoughts, big or small, are valuable. If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.
For any questions or concerns, send mail to test@lists.fedoraproject.org.
Providing feedback
- Gwjasu - I like ____ about the new ____ process
Adding feedback is fairly straight forward. If you already have a Fedora account ...
- Login to the wiki
- Select [Edit] for the appropriate section below.
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Otherwise, if you do not have a Fedora account, follow the instructions below ...
- Select the appropriate page for your feedback...
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Feedback
Things that went well
- jlaska - Beta released on time - Without a strong turn-out from the fedora-qa community, there is little chance that we would had the test results needed in order to properly assert that the beta release criteria had been met.
- jlaska - Test Matrix - identifying concerns with the Fedora 13 test matrices, and implementing improvements for Fedora 14 worked really well. I really like to revised organization and cleaning of obsolete tests in the installation matrix (see QA:Fedora_14_Install_Results_Template).
Could have been better
Fedora 16 alpha (Gnome) fails to complete properly (and does not give compliance warnings) when an NVIDIA Quadro FX3400SLI (HP Supplied) is installed in an HP XW4600 Workstation. (Works fine with NVIDIA GeForce similarly scaled cards). Under Fedora 15 (Gnome) the Quadro card reports no support under Gnome 3. - Cheers - reynolds@mira.net
- jlaska - Beta RC delayed - Beta RC candidates were delivered late, as there were 2 unresolved beta RC blockers (see below). Specific to the following two issues, it's not clear what can be improved/corrected next time?
- jlaska - Fixing more than blockers - For F-14-Alpha, a new anaconda with 2 blocker bug fixes was requested, but the new version contained many more fixes. Question, did the additional bug fixes break anything?
- Recommendation - Early creation of f14-{alpha,beta,final} git branches for anaconda to allow for more control over commits.
- jlaska - Root cause analysis - on why the following issues were discovered late
- RHBZ #627789 - several users reporting problems installing from HD ISO due to this bug. However, testing RC3 showed no problems installing HD ISO ... there is a disconnect somewhere between the use case exercised and the test case? (see docs follow-up)
- RHBZ #638091 - Kernel panic at boot after glibc update - Updating to
glibc-2.12.90-13prelinkcommand ran, and then it would never boot and all commands would segfault on a running system. The issue was initially given positive proventester feedback, but was then reverted after the tester found the problem. The good news, the problem was discovered in updates-testing before it landed in updates. However, it would be good to have some documented test procedures for a subset of packages to guide proventesters
- jlaska - Virt Test Day - Participation in the test day was low and not what we had hoped for. We identified this problem during the Fedora_13_QA_Retrospective, what can be done to provide meaningful public test events for Virtualization?
- jlaska - KDE bugs - are KDE-only bugs release blockers? It's unclear how to handle this or what FESCO intends (see RHBZ #641338).
- jlaska - Wiki Test Matrix - we've hit the maximum threshold for mediawiki parser function calls, Category:Pages_with_too_many_expensive_parser_function_calls (for specific example, see Test_Results:Fedora_14_Beta_RC3_Install). Might be able to modify Template:Result to reduce the parser function calls, but not sure what else to do. Switching to a formal test case management system for this specific issue seems like overkill, but certainly an option.
- jlaska - Proventesters - The proventesters sign-up and instructions are awesome, but I think we have a gap around providing specific instruction for certain packages (kernel, glibc etc...). Can we outline a process and some wiki structure to encourage mailing list discussion and drafting of package-specific test procedures (e.g. [[:Category:Test_Cases] or Category:Test_Plans)?
Wishlist
- I want a pony
Recommendations
After enough time has been given for feedback, the QA team will discuss and make recommendations on changes to prioritize for Fedora 14. This section organizes and lists the recommendations.
In order to coordinate efforts, and measure effectiveness of recommendations, please record and track any action taken in the Fedora 15 roadmap in the QA TRAC instance.
Recommendations will arrive typically after problems have been diagnosed and improvements reviewed. This typically occurs after the release. Stay tuned ...