From Fedora Project Wiki

< User:Roshi
Revision as of 17:00, 7 January 2014 by Roshi (talk | contribs)

This content is relocated
The content in this page is from the now-defunct Bugzappers area of the wiki. All the information regarding triaging is correct, but there is no BugZappers group. Any questions should be sent to test or fedora-qa[?].

Preparing to Triage

Choose Bugs Wisely

Most triage is done on bugs filed in a specific component. For lists of components and associated bugs, see BugZappers/Components and Triagers.

Please do not touch bugs with a product that is not "Fedora" (especially Red Hat Enterprise Linux bugs).

Please do not triage bugs filed against the "Package Review" component, which usually have "Merge review" in the summary. These are handled as part of the Package Review Process.

Use the Best Tools

Some people find the Package Database (pkgdb) interface to Bugzilla preferable to the standard Bugzilla interface. For example, to use pkgdb to inspect open firefox bugs, visit: https://admin.fedoraproject.org/pkgdb/acls/bugs/firefox. For a complete list of packages, refer to link.

It is also recommended that you install the Bugzilla Triage Addon in your web browser. See BugZappers/Tools for this and other helpful tools.

Stock responses are essential for any triager.

Understand Bugzilla

Checklist for NEW Bug Triage

This checklist is intended for NEW bugs. With some practice, you should be able to use the bold headings as reminders, and skip the details in most cases.

1.) Is this a bug?

  • Sometimes there is no malfunction or design flaw; the reporter simply needs help configuring or using software. These reports and anything else that's not a bug or feature request should be set to CLOSED/NOTABUG with a polite message directing the reporter to a different forum.
  • Requests for new packages to be added to Fedora should be CLOSED/NOTABUG and added to the Package maintainers wishlist. See the stock response.
  • Feature requests for existing packages are OK; just add in the keyword field: FutureFeature

2.) Is it assigned to the correct product and component?

  • See BugZappers/CorrectComponent if you need technical assistance.
  • If this is another Red Hat product like Red Hat Enterprise Linux, change the Product field.
  • If this is software running on Fedora but not distributed by the Fedora Project mark the bug CLOSED/CANTFIX and if possible give a pointer to the correct forum. See ForbiddenItems for a list of software that is intentionally not shipped with Fedora. There are stock responses to deal with these reports.
  • If it's obvious the component is wrong, change it.
  • Sometimes it cannot be determined which component needs to be changed to fix a bug until the source code is examined. For triagers, it's fine if the assigned component is only a reasonable guess. Feel free to ask the package maintainer if the bug has been correctly assigned.
  • The component 0xFFFF is sometimes chosen incorrectly simply because it is the first one on the list.

3.) Is this a duplicate?

  • If so, set the least helpful report to CLOSED/DUPLICATE with the bug number of the most helpful report and this stock response.
  • If it's not obvious why a bug is a duplicate of another, add an explanation in a comment for the benefit of the reporter.
  • See BugZappers/FindingDuplicates for help if you need it, but basically:
    • Be familiar with Bugs/Common.
    • Either browse or search for similar bugs filed against the same component.
    • If there's any possibility a different component is involved, do a keyword search for similar bugs filed against all Fedora components.
    • If you find a bug that this bug depends on, or which depends on this bug, mark this in the appropriate fields.

4.) Are there related bugs?

  • If fixing another bug would help this one, or vice versa, add comments to both bugs explaining why.
  • If you are certain about this and know which bug should be fixed first, mark that one bug depends on the other.

5.) Are multiple problems reported in the same bug?

6.) Is the version number reported correctly?

  • The number in the Version field is the Fedora release version (9, 10, Rawhide), not the RPM version number.
  • The exact version number of the RPM should be included in the comments from the reporter.

7.) Is there sufficient information for the developer to reproduce, diagnose, or fix the problem?

  • If not, you can either:
    • Ask for more information from the reporter. Leave the status as NEW, select the box next to "Need additional information from" and select "reporter" from the pull-down. Be sure to leave a friendly note in the comment field describing what the developer needs and how to get it.
    • Supply the information yourself. (This usually involves attempting to reproduce the bug; see Optional Step 12.)
  • See BugsAndFeatureRequests for details on what "sufficient information" means for each type of bug. This generally involves:
    • The exact version number of the affected RPM.
    • Clear steps describing what happened or preferably exactly how to reproduce the bug.
    • The text of any errors or warnings produced (verbatim, if possible).
    • Log, troubleshooting, or configuration files if appropriate (depends on the component).
    • Stack trace if a crash occurred.

8.) Is the summary appropriate?

  • Feel free to improve the one-line summary if what the reporter has written is overly broad, narrow, confusing, or there are spelling errors.
  • Be careful not to subvert the intent of the reporter.

9.) What severity should I set?

As part of triage, you should set the severity field for the bug report. Most bugs should be medium, however the criteria to evaluate bug severity is listed below:

  • Urgent: the bug makes whole system unusable (or it is a security bug, which is per definition urgent)
    • if this is a Rawhide bug, also consider if it should be set as a blocker or target bug; see below
  • High: the bug makes the program in question unusable, or the issue is a serious violation of the packaging guidelines (e.g. license issue or bundled library)
  • Medium: a real bug which makes program more difficult to use, at least part of the program is available; possibly workarounds are available
  • Low: anything else - cosmetic issues, corner cases with unusual (non-default) configurations, etc.

As a caveat, you should usually not use the urgent setting for hardware-specific bugs: a bug which causes the entire distribution to be affected but is restricted to a single specific type of hardware should usually be set to high. For instance, if a bug prevents X.org working correctly on a single particular graphics chipset, set the high severity, not urgent.

Use your judgement in setting this field, but use the high and especially critical settings sparingly, and try to be consistent both with your use of the field over time, and with the use of the field by other triagers.

10.) Should this be a blocker or target bug? (Rawhide bugs only)

Top priority bugs that either must be fixed before general public release (and thus will delay the release if not fixed) should be added to the "Blocker" tracker for the release. Important bugs that should be fixed before general public release (but won't block the release) should be added to the "Target" tracking bug for the release. See BugZappers/HouseKeeping/Trackers for links.

11.) Is it a bug trivial to fix?

If you think that the bug is trivial to fix, please add the "EasyFix" keyword. This enables provenpackagers to track for bugs that they may easily help to fix and prevents such bugs from starving a long time in Bugzilla. Examples of what could be considered as EasyFix bugs are: missing requires (or similar minor updates to the SPEC files), bugs with a simple patch attached, and spelling errors.

12.) Is there a patch already attached?

If so, make sure the word "Patch" is present in the Keyword field.

How to Mark as Triaged

If you have completed the entire checklist for a bug, you should mark it as triaged so that others will not duplicate your efforts.

For Fedora 12:

  • If the bug now has all the information the maintainer needs, mark it as ASSIGNED. If you are waiting on information from the reporter, leave it as NEW and check the "Need information from" box and set the pulldown appropriately.
  • Either way, add "Triaged" to the Keywords field.

For Rawhide, Fedora 13, and later releases:

  • Add "Triaged" to the Keywords field. Do not change the bug's status. If information is missing from the report, check the "Need information from" box and set the pulldown appropriately.

Closing Bugs

Many bugs are fixed upstream or unintentionally during a seemingly unrelated rewrite. If someone who has experienced the bug can confirm that is is fixed by an update to the version of Fedora is it reported against, mark it CLOSED/ERRATA.

Normally when a package maintainer intentionally fixes a bug, this is recorded in the build system, and Bodhi should automatically close the bug when the update goes live. (But this does not happen for Rawhide bugs.)

How to Handle Bugs in Multiple Versions

If you or anyone else is experiencing the bug in a more recent version of Fedora or Rawhide than originally reported, note this in a comment and change the version number. This prevents the bug from being closed when the older version hits its End of Life.

Many bugs are fixed in the code latest Fedora version, and then must be intentionally backported to earlier versions. It is up to the package maintainer to decide whether a backport will be produced, based on the time they have available to work on the problem, the duration and severity of the impact on users, the difficulty of implementing the backport due to changes in the code over time, and the risk of creating new problems with the attempted fix. It's OK if minor bugs aren't fixed in older versions. They will soon reach their End of Life and all users will upgrade to a version in which the bug is fixed.

Not creating unnecessary copies of bugs at triage-time avoids splitting the discussion, and avoids cluttering Bugzilla and creating extra work for you and everyone else.

If you or a reporter would like to actively advocate for a backport to an earlier version, you can make this request in a comment in the same bug.

If a package maintainer closes the bug without making a backport (perhaps overlooking the request) or this is an important backport (such as a fix for a security bug), file a separate bug against the specific version, but mark it as depending on the main bug. Direct others to bring discussion about how to fix the underlying problem to the main bug.

Optional steps

12.) Help reproduce or isolate the cause of the bug.

The below steps might be quick and easy, or be quite time consuming, depending on the type of bug. They are not necessary before marking the bug ASSIGNED, and it is recommended you not spend too much time on an individual bug if you have a large number to triage. However, any contributions you choose to make (which can be made even by non-programmers) are helpful and extremely welcome if you have the time.

There are two big questions that are extremely useful to answer:


13A.) Is the bug due to a specific configuration or hardware?
13B.) Has the bug already been fixed?


If this is a non-reproducible event even for the original reporter, like a crash or intermittent failure, you will need to rely on people using the program for everyday use, and reporting the problem when it occurs. The causes of crashes are sometimes hard to pin down, and they are often fixed as a byproduct of other changes in the code.

  • Be sure the reporter is running the latest update for their Fedora release.
  • If you do not see bug reports against more recent release, you can offer the reporter the option of upgrading to a new Fedora release (which is often a large pain, but which they might prefer).
  • The problem might be due to the specific configuration or hardware used by the reporter, so there is no guarantee than an upgrade will fix it. But running more recent code makes it more likely developers will track down the cause.


If this is a reproducible bug for the original reporter, you can try to experience the bug yourself. You may need to attempt reproduction with multiple releases to answer both questions 13A and 13B. The order in which you choose to do so depends on which release is the most convenient for you to use, or which question you wish to answer first.

  • The only reason to test an older release Fedora than the bug was originally reported in is if it's important enough that a patch should be backported to the older release. Otherwise, the older release can simply be allowed to reach its end of life; as long as the bug is fixed in future releases, it will eventually go away for everyone. Backporting can be time-consuming for developers, and this should be weighed against the impact for users.
  • If the bug can be reproduced in the same Fedora release as the reporter, or any later release, this is evidence (though not conclusive proof) that the bug is not due to a specific configuration or hardware.
  • If the bug can be reproduced in the same Fedora release as the reporter, but not in later releases despite trying, this is strong evidence that the bug has already been fixed. Offer the reporter the option of upgrading to a newer release (or waiting until the next release if it is only fixed in Rawhide). The bug should stay open; you can add a comment leaving it up to the package maintainer whether or not they wish to backport a fix. If not, they will close the bug. If there is consensus to close the bug, it should be marked as CLOSED/CURRENTRELASE, CLOSED/RAWHIDE, or CLOSED/NEXTRELEASE as appropriate.
  • If a bug is confirmed fixed by an update in the same release, mark it CLOSED/ERRATA.
  • If the bug cannot be reproduced in the same Fedora release as the reporter, this is strong evidence that the bug is due to something different in the environment of the reporter.
    • Make sure that the reporter is using the latest updates for that release, and compare RPM version numbers.
    • Make sure that you are using exactly the same method to reproduce the bug as the original reporter.
    • If you suspect user-specific state (configuration, cached data), ask the reporter to create a new Unix user and attempt reproduction with that account.
    • If you suspect machine-specific state (configuration, cached data), ask the reporter to attempt reproduction with the a "fresh" RPM installation. This can be done by un-installing the problem RPM, moving any cached data or configuration files aside (for possible later dissection) and re-installing the RPM. Obviously caution should be used if this is a vital component or if "--nodeps" is used to manually remove the RPM. There is no guarantee this will fix the problem, or that it will not make things worse, so be sure the reporter knows what they are getting into. The reporter may not want to do this if they need this software for everyday use or do not wish to risk disturbing their configuration.
    • If you suspect a hardware-specific problem, you might request a Smolt profile. The reporter may need to "yum install smolt" if it is not already installed, then run "smoltSendProfile" and add the public URL provided to the bug in a comment.

The latest updates for all releases can be found in the Koji Build System, even before they appear in yum repositories.

14.) Has this already been reported upstream?

  • Do a search in the upstream Bugzilla or mailing list, if they exist.
  • If you find a duplicate report, link the Fedora bug report to the upstream one, but leave the Fedora report open.
  • Leave it up to the Fedora package maintainer to decide whether they should fix the problem themselves (which will likely involve submitting a patch upstream) or leave it in the hands of the upstream developers and mark the bug CLOSED/UPSTREAM.

Upstream bug reporting systems:

Followup

If you set a NEEDINFO flag in a bug, it is good practice to return after 30 days to see if the reporter has supplied the requested information. (See the NEEDINFO checklist, below.) If for some reasons you do not do so, other triagers who are only looking at NEEDINFO bugs can also do this.

Checklist for EOL Bug Triage

For bugs filed against Fedora releases that have reached their End of Life (EOL):

  • If the bug appears to be occurring in a more recent (non-EOL) version, update the version number and leave the bug open, but advise anyone still running an EOL version of Fedora that that they should upgrade to receive important security fixes.
  • OPTIONAL: You could attempt to reproduce the bug yourself in a recent Fedora version, but beware that many bugs are fixed between releases (though not all). Be sure to first check if the bug has already been reported.
  • Otherwise, mark the bug CLOSED/WONTFIX and add the EOL stock response.

Checklist for NEEDINFO Bug Triage

Reviewing bugs with NEEDINFO:reporter flag set:

  • After the NEEDINFO flag has been set for 30 days with no response, send a reminder:
This is just a reminder that if the requested information is not supplied within 30 days, this bug will be closed. Thanks for your assistance in improving Fedora.
  • After the NEEDINFO flag has been set for 60 days with no response, mark the bug CLOSED/INSUFFICIENT_DATA.
  • Alternatively, you may also attempt to provide the requested information yourself, or send e-mail to fedora-test-list asking for testers to assist.
  • Feel free to use your judgment with regard to the appropriate timeframe to send reminders or close the bug.

Easier Testing on Different Fedora Versions

Not everyone has three different computers each running a different Fedora release. Reporters, triagers, and developers can use LiveUSB instances for easy testing of a different release on the same machine, with only a reboot. To do this, follow the instructions on the How to create and use Live USB page using the "data persistence" option. Boot the LiveUSB instance, and run "yum update" as normal to test the latest packages. (Not recommended with Fedora 11 Alpha due to a large number of updates.)

To test a different release using an emulator, see Testing/qemu.

General Advice

  • Please remember to be polite when triaging bugs; we would like reporters to continue doing so in the future.
  • Avoid requesting information of reporters that isn't really necessary; this is obviously frustrating for them. At least be sympathetic if you make a request that might be inconvenient to fulfill.
  • Avoid requesting unnecessary re-testing; this is also frustrating for reporters. Often the best thing to do is to try to reproduce the bug yourself. (Obviously this is not applicable to intermittent crashes, hardware-specific bugs for hardware you don't have, etc.) If this fails, you can reporter to the tester either that they didn't provide enough information, that the bug is fixed in a newer version, or that the bug is hardware or configuration dependent. This information is useful to the reporter (provides motivation to upgrade, retest, or dig deeper) and the package maintainer (makes it easier to track down the problem).
  • Avoid marking a bug as a duplicate that isn't really the same. If you don't have the technical expertise to be certain, just add a comment with the other bug number, and say it's a possible duplicate.
  • If the developer has commented on the bug or filed it themselves, if it's not a duplicate, it's probably OK to automatically mark it as ASSIGNED. (If more information was needed, they probably would have requested it themselves.)
  • If you aren't sure about something, feel free to ask in #fedora-bugzappers, or leave a comment in the bug for the package maintainer. Be especially careful about closing bugs.
  • Add yourself to the CC: list of bugs you triage.
  • Please read carefully, and think before you click.
  • Use BugZappers/StockBugzillaResponses as appropriate.

Further reading

  • Painless Bug Tracking by Joel Spolsky … Joel is owner of a company producing proprietary issue tracker. We cannot use his code, of course, but he knows a lot about bugs, and this article contains really the foundation of the trade.
  • Talks by Luis Villa … all is worthy to be known about bug triage in the open source world comes from him.
Retrieved from "https://fedoraproject.org/w/index.php?title=User:Roshi/QA/Triage&oldid=366076"