QA:Update feedback guidelines

Introduction

This page presents general guidelines for providing feedback on candidate updates, using the Bodhi system. It explains how you should determine whether to provide positive, negative, neutral or no feedback on any given candidate update. For instructions on obtaining and installing candidate updates, see QA:Updates Testing. For information on the mechanics of actually posting feedback, see the Bodhi page. For a convenient tool which provides a command-line interface that allows you to quickly leave feedback on all or some candidate updates installed on your system, see the Fedora_Easy_Karma page.

Positive feedback

Usually, you will be able to post positive feedback on an update. If you installed one or more candidate update(s), and found that the updated package seemed to work as expected and your system as a whole continues to work as normal, and you do not encounter any of the situations below, you should leave positive feedback and note that you were able to use the package successfully and did not notice any significant problems. Please read all sections below before posting positive feedback.

Neutral feedback or no feedback?

Neutral feedback - 0 karma, marked in the web interface as Untested - is intended to be used in the case where you have a significant comment to make on the update, but you cannot with confidence post positive or negative feedback. For instance, you noticed what you think may be a bug, but you are not sure. It is not meant to be used in the case where you have nothing useful to contribute; active feedback is not required in this case. If you are using a tool like fedora-easy-karma and it presents an update you cannot confidently provide any feedback on, simply leave no feedback at all, do not leave neutral feedback with a note saying 'not tested' or similar. You can skip an update without leaving any comment in fedora-easy-karma by entering anything except -1, 0 or 1.

Major bugs

If you identified any serious problems in your testing and were able to identify the update responsible, check that the bug is new - that it did not exist in the stable version of the package - and then post negative feedback for that update. Do not post negative feedback on an update simply because it still contains a bug that already existed in the previous version of the package. Bear in mind that the function of negative feedback is to prevent an update from being released; it makes no sense to file negative feedback on an update which is no worse than the previous version of the package.

If possible, please file a bug report on the problem and link to the bug report in your feedback message. A good feedback message quickly and clearly identifies the behavior change and the cause, if you were able to determine it.

Minor bugs

If you identify a problem in an update which is minor in nature, use judgement to decide whether to post negative feedback. Consider whether overall the update constitutes an improvement on the situation with the previous package - for instance, if an update fixes a significant security package but introduces a minor cosmetic bug, negative feedback is probably not appropriate. Instead, post neutral or positive feedback with a note of the issue encountered (and a link to a bug report if appropriate).

Previously reported bugs

If your testing uncovers no problems but you see that another tester has identified a serious problem with the package, please try to replicate their problem, and post negative feedback if you are now able to confirm it. If you are not able to confirm the problem but you suspect this may be because you cannot recreate the necessary conditions, please post neutral feedback noting that you were unable to duplicate the problem. Only post positive feedback if you are sure your testing indicates the other reporter's negative feedback is a mistake.

Update does not fix a bug it claims to

If you find an update does not fix a bug it claims to fix, this is not usually a case where you should file negative karma. Only file negative karma if that is the only change in the update. If an update claims to fix five bugs, but only fixes four of them, it is not helpful to post negative karma as this may result in the update being rejected, which does not help those suffering from the bug that wasn't fixed, and hurts those suffering from the bugs that are fixed. When you test an update that claims to fix a particular bug and doesn't, but does not have any of the issues listed as meriting negative or neutral feedback above, please leave positive feedback with a note that the bug in question is not fixed, or neutral feedback with such a note if the issue prevents you from otherwise properly testing the update.

Unfamiliar packages

If you are not sure what the component does or how to test it, do not post positive or negative feedback. For critical path updates, if general tests of booting, network functionality and update functionality identified no problems, it is fine to leave a neutral feedback message noting that you were able to boot the system and perform critical path tasks with the update installed. This is generally not useful for non-critical path updates, however: please only comment on them if you are familiar with the package and able to test it directly.