Fedora Bugzilla

The Problem - (Almost) Too Many Open Bugs to Manage!

Fedora has a proliferation of open bugs (13,588 as of 2008-01-06), so much so, that we are getting to the point (or may well have reached it) that it is becoming unmanageable. While the number of bugs has grown, our bug triage community has become dormant, further reducing our ability to maintain control. Fortunately it appears that we have not reached the point where we can no longer manage releases under development--thanks largely to the help of tracker bugs. However, it does not seem logical to believe that we can continue on as we are and not do long term damage to the Fedora project.

The time to act is now!

By not addressing this arms race of bugs we put a brighter Fedora future in jeopardy:

We alienate community members that report bugs, but (never) do not see a timely response.
As fewer people report bugs, the quality of Fedora releases has a higher change of declining
We discourage community bug triagers from joining a situation where the results of their work cannot be seen because bug activity is lost in the ocean of other bugs.

Following are a collection of ideas--some originally mine and many from others--scraped from various email threads (fedora-advisory-board, fedora-devel, fedora-test, fedora-marketing, Fedora-triage), blog posts, and IRC conversations about this topic.

The goal of this Wiki page is to provide a place where we can review all these ideas - and others - and decide on a plan to clean up the bug backlog and handle new bugs.

Bugzilla/Triage Wiki Pages

Existing wiki pages about Bugzilla and Bug Triage:

These pages need to be:

Cleaned up and made relevant
Normalized into as few pages as possible
Streamlined to be accurate, clear and readily understood
Able to engage new participants in a way that makes them feel valuable and rapidly effective--beyond the undesirable quadrants of the graph

Bug triaging isn't always fun
rise above the "suck threshold" quickly (this applies to the entire Fedora Project!)
http://headrush.typepad.com/creating_passionate_users/2005/10/getting_users_p.html
"Kicking Ass is More Fun"
http://headrush.typepad.com/creating_passionate_users/2005/06/kicking_ass_is_.html
We have to focus on helping people learn how to triage
We have to create a way for them to feel successful and that their valuable time is being well spent

Breaking Down the Bugzilla Situation - And What We Should Do About It

Past
handling EOL releases
When: eventually
Present
day to day triaging of incoming Fedora 7, 8, and rawhide bugs
When: starting NOW
Future
documenting how we will triage bugs for future releases
add important changes to bugzilla at the GA to the Fedora schedule
1. New version to bugzilla to be ready on release day
2. Changing the Fedora Product description to reflect the correct releases documenting how we will triage bugs for future releases and important changes to bugzilla at the GA of a new release
3. Moving rawhide bugs (see subsequent discussion)

Most pressing issues in order of priority:

Start a concerted consistent triaging process for incoming bugs for Fedora 7,8 and Rawhide
Triage incoming bugs in an intelligent, scalable way
Close out bugs open against unsupported releases

13,500+ Open Bugs

8 Release of Fedora since its birth
Rawhide (previously known as development is ongoing
There is no feasible way to review every single bug. We could, but is that really the best use of everyone's time?
What is our plan of attach to bring this down to a reasonable number?
After we bring the bug count down to a reasonable number:
1. What will we do on a consistent basis to make sure it doesn't happen again?
2. How will we cultivate a community of people to help keep things under control?

Mass Close of EOL Versions

Closing all bugs for unsupported releases
start the closing process after three months of consistent community triage work
demonstrate that triaging work is sustainable by the community
don't want to unnecessarily alienate community twice if we can't follow through
<= Fedora Core 6
Closing all bugs opened against rawhide that were opened more than 1.5 years ago
Possible process could look like
1. write proposal and run past FESCo for sanity check and support
2. establish criteria for which bugs to select
3. start with flipping to NEEDINFO_Reporter explaining what we are doing and requesting feedback
4. flag/tag bug as on the "we're going to cl0s3 u' list
5. wait fixed duration of time--2 weeks?

Life Cycle

lifecycle of a bug document
What are the bugzilla statuses Fedora bugs should go through and in what order?
Can this be enforced by bugzilla to reduce needed human intervention
What is our policy for accepting bugs against unsupported releases?
If we decide to disallow bug filing against unsupported version can bugzilla enforce this?
Can we utilize bug priority and severity in a useful way that helps developers and triagers?

Triage

community needs to be restarted
existing wiki pages updated
rewards: t-shirts, hats, stickers, OLPC, All expense paid vacation-I mean trip- to FUDCon, Triager of the year
How do we effectively identify duplicates?
This has been pointed to as a critical area. Where can we find and then document guidance on best practices here?
Effectively start to triage incoming bugs on a daily basis.

Future

Create (or update) the lifecycle of a bug
Change the version of all rawhide bugs at the time of GA to the version number to be GA
For example, all current rawhide bugs become Fedora 9 bugs at the GA of Fedora 9
This anchor to the closest released version.
Clears the deck after each release under development and makes it easier to sort through new bugs for the next release under development.
Need to agree on a policy
Add to Fedora schedule

Structuring Bug Days

My first experience attempting to participate in a bug day was less than optimal.

It wasn't very organized--I was simply told to "look at bugs in packages that are interesting to me"--this seemed really suboptimal and turned out to be so. As a result:
1. I was looking bugs that other people had already triaged
2. It was completely random as to whether I was spending my time on bugs that mattered
3. I wasn't that enthusiastic about joining again because I didn't feel like it was a valuable use of my time
The fact that the bug day was unorganized didn't appear to bother some people, but is it worth losing other people that might feel their time is being wasted?

We need a methodology

Which bug states should be queried
Where do we describe what state a bug should be in?
When should a bug be in "NEW" vs. "ASSIGNED?"--seen a few cases where there were 3 or 4 comments and evaluation of the problem, yet the bug was NEW. Should a triager change to assigned?
- If a bug is "fixed in rawhide" or "fixed in version x.x.x" should the bug be in MODIFIED or CLOSED?
Does a bug have to be verified first to be closed?
Is it worth looking at NEEDINFO bugs?
If reviewing a bug in NEEDINFO how long is "long enough" before requesting more feedback or closing?
We should probably be explicit about this in the boilerplate--"We will take ______ action if we do not see a response within ______ days."
What state should bug be in if comments say "I've filed this bug upstream as ______ " __
How do we structure bug days so that we are intentional about making sure two people are looking at the same bug

Tools

Existing Tools

Fedora Bugzilla Start Page

What is the status here?
Where is the existing design and/or code?

Bug Buddy

What is the status here?
Where is the existing design and/or code?

Apport

What is the status here?
Where is the existing design and/or code?

New Tools Needed

Reports/graphs/metrics that give a clear and simple status report on how we are doing
a simple web page or email notification--no perl or Python scripts or complicated SQL scripts
other ideas:
trend of total open bugs by month
# of bugs closed each month
# of new bugs opened each month
ascending ranking of the names top 50 packages with the most bugs and how many open bugs.
aging report showing days-outstanding--like an A/P (accounts payable or receivable aging).
Essentially how long individual bugs that are currently open, have been open--broken down into different categories by days outstanding and graphed
y axis == number of bugs
x axis == buckets/days out standing (0-30, 31-60, 61-90, 180, 365, > 1 year)

Bugzilla does basic charts - one example for F7 kernel bugs here

FUDCon Discussion Ideas and Questions

Can we query rawhide bugs based on the date they were opened && last activity?
1. Flag them in a particular way
2. Post comment requesting testing feedback against the latest package version by a certain date
3. Auto-close all bugs where no action has been taken
How many NEW bugs are coming in everyday?
Could we have auto-queries or notifications for all bugs opened against unsupported bugs?
Create boilerplate response for all unsupported versions--triagers set status to CLOSED/WONTFIX and encourage reporter to reopen if the bug exists against a current version
Engage universities to own a section of bug triage
What about the college or professor that posts to the Fedora list requesting that people run debug version of certain programs? Could they own triage for those packages?
Do we kill fedora-triage-list for the time being?
Send triage related email to an established list with a common purpose where traffic and readers are already established
Could we make use of the "QA Contact" field to assign triagers?
If we want to scale--maybe assign groups of packages to have QA Owner set to a certain mailing list
What if a bug exists against two different supported Fedora versions?
Open two bugs (clone)?
Other?
How do we encourage or guide better bug filing?
Better process for filing/moving/cloning bugs upstream
Create a project plan with some milestones we want to achieve
corresponding to having certain parts of these processes completed or in place by Fedora 9 GA
Do a test of the new triager signup steps to make sure new triagers can signup for Fedora from scratch by following our existing procedures
This should verify that we have the right FAS groups, permissions, etc.

Search

User:Poelstra/ImprovingBugzillaF9

Contents