Overview
A proven tester ensures that updates to critical path packages do not violate the Fedora Release Criteria and do not break any critical path actions. The tester will provide feedback through Bodhi and Bugzilla in the form of karma and bug reports. This feedback minimizes the risk of broken updates as their stability is assured before they are pushed to the community at large.
Responsibilities
Feel free to vary the details as suits you, but please bear in mind the advice on what types of feedback to leave in what cases.
Find and install package updates for testing
Enable the updates-testing repository, if you haven't already, and do a full system update each day. It is of course fine to quickly install one specific critical path update and test it in isolation, if you are particularly interested in ensuring the update is tested promptly, or the package maintainer sends out a request for testing.
Check for major regressions
Perform a quick functionality test to ensure the updated package is working as intended. If the package is an application, run the application and check that basic operations work. If it is a library or other shared component, run an application which uses the component and ensure that that works.
- After doing the update, reboot the system and log in to the desktop (assuming you can). If you observe any problems or changed behaviour, take a detailed note and try to identify the package responsible for the change.
- Ensure your network access is working as usual. If not, try to identify the cause.
- Check that the update application will run, although you now likely will have no updates remaining to test immediately whether it works. If not, try to identify the cause.
Investigate & provide feedback
If the component works as expected, post positive feedback. If you identify a major problem, post negative feedback. If you identify a problem which is minor in nature and does not impede the actual critical path functionality, please do not post negative feedback. Post neutral or positive feedback with a note of the issue encountered (and a link to a bug report if appropriate).
- Use
fedora-easy-karma - Pay particular attention to updates whose description notes that they are critical path updates. If you identified any serious problems in earlier testing and were able to identify the package responsible, post negative feedback for that package. If possible, please file a bug report on the problem and link to the bug report in your feedback message. A good feedback message quickly and clearly identifies the behaviour change and the cause, if you were able to determine it.
- If you are not sure what the component does or how to test it, do not post positive or negative feedback. If the above general tests of booting, network functionality and update functionality identified no problems, it is fine to leave a neutral feedback message noting that you were able to boot the system and perform critical path tasks with the update installed.
- If your testing uncovers no problems but you see that another tester has identified a serious problem with the package, please try to replicate their problem, and post negative feedback if you are now able to confirm it. If you are not able to confirm the problem but you suspect this may be because you cannot recreate the necessary conditions, please post neutral feedback noting that you were unable to duplicate the problem. Only post positive feedback if you are sure your testing indicates the other reporter's negative feedback is a mistake.
