From Fedora Project Wiki

< QA‎ | Meetings
Revision as of 20:28, 15 April 2009 by Jlaska (talk | contribs) (Updated minutes)

Attendees

  • Will Woods (wwoods)
  • Adam Williamson (adamw)
  • Jóhann Guðmundsson (viking_ice)
  • John Brown (tk009)
  • Jesse Keating (f13)
  • James Laska (jlaska)

Previous meeting follow-up

  1. [jlaska+adamw] mediawiki semantic update (packaging and hosting)
    • REVIEWED - 490001 - Review Request: mediawiki-semantic - The semantic extension to mediawiki
    • UNDER REVIEW - 490171 - Review Request: mediawiki-semantic-forms - An extension to MediaWiki that adds support for web-based forms
      • User:tibbs posted some additional concerns for mediawiki-semantic-forms around licensing which I haven't followed up on yet
      • fedora-infrastructure request sent for fedoraproject.org/wiki database copy for testing
  2. [jlaska] - reach out to pmuller on packaging rhtslib
    • Discussed briefly with pmuller the current status of rhtslib packaging, will reach out again this week
  3. [jlaska] - improve the kickstart file used for generating test day live images
  4. [adamw] - send details of nss rawhide issue to warren for posting to rawhidewatch.wordpress.com
  5. [jlaska] - talk to John Poelstra about adding a few F11Blocker review meetings to the schedule
  6. [adamw] - review Test Day X11 bugs to ensure they are represented on F11Blocker
    • noveau bugs - I was looking at nouveau bugs, then checked in with darktama and he says he's already doing a review of all of them himself
    • radeon/intel bugs - I will talk to matej and francois about
    • I am a bit worried about radeon, there are rather a lot of open reports on it, several looking quite serious
  7. [wwoods] - review pulseaudio/alsa bugs and work with lennart and jaroslav for F11Blocker representation

Autoqa update

    • Will noted it would be good to have a test day for pulseaudio, but consensus was there is no room in the test day schedule and the hardware people are having issues on is very common
  • Next steps:
    • continue improving existing test reporting
    • interim goal of sending automated test result mails to autoqa-results
    • wwoods suggested there are fixes that he would like to get into upstream beaker
    • reach out to pmuller for information on packaging rhtslib

F-11 Blocker Bugs

James Laska asked to brainstorm on a way to open up the process of assessing milestone blocker bugs. Adam noted he has been engaging the bugzapper team on this topic but they have expressed concerns as they feel they might not do it right. Will Woods acknowledged that expanding and clarifying release criteria (or having separate blockercriteria) might be a good idea.

There are several resources available now:

Adam noted I guess if we adopted that system going forward, triage would naturally feed into blocker bug evaluation

Jesse questioned whether exposing and actually trusting severity/priority in bugs is worthwhile ... Every person's pet bug is high priority high severity.

Jesse asked what success would look like for a trial run?

  • no revert wars, positive feedback from maintainers looking at their bug lists
  • community participation in the escalation of blocker bugs

Jesse offered concern that the first instinct of many people will be to think that QA will want to force what people will work on.

  • James suggested QA just provides data/guidance on the bugs. How the list acted on is a different topic.

Adam summarized by noting he is working on a draft for developer feedback, this process is intended to advise developers (not dictate). If it works great, if it doesn't we'll kick it to the curb, and we can evaluate down the road how severity/priority and blocker lists are/should be interacting.

Jesse suggested that Triage and setting of priority/severity can certainly help the maintainer make that decision, so long as they have the ultimate say. Jesse also concluded that whatever helps us or maintainers find the critical issues sooner rather than later will help

Jóhann added that this will never work. Adam encouraged optimism.

Open discussion

There wasn't enough time for open discussion in the meeting. Please send thoughts/ideas to fedora-test-list@redhat.com.

fedora-qa-bookmarks package?

Cool idea proposed by wwoods - a Fedora QA bookmarks package (installable for test days, rawhide testers, live images etc...). Jesse Keating noted it's a challenge to have this co-exist with the fedora-bookmarks package.

Upcoming QA events

Action items

  • [jlaska] - announce live image create wiki update to fedora-test-list
  • [adamw] - discuss with mcepl and francois about radeon and intel F11Blocker bug status
  • [adamw] - post to fedora forums asking for feedback on pulseaudio issues
  • [jlaska] - schedule autoqa discussion w/ wwoods and jkeating - is there any work we can do prior to F-11 GA?
  • [adamw] - send a priority/severity definition draft to fedora-test-list for review before sending to developers/maintainers

Next QA meeting

The next meeting will be held on 2009-04-22 16:00 UTC

IRC Transcript

--- | jlaska has changed the topic to: Fedora Quality Assurance Meeting | init Apr 15 12:01
* | wwoods here Apr 15 12:02
jlaska | wwoods: Hey there Apr 15 12:02
* | jlaska invites adamw Apr 15 12:02
adamw | sorry, here now Apr 15 12:02
jlaska | hey there ... no worries Apr 15 12:02
jlaska | I've got wwoods and adamw ... anyone else joining the QA meeting, please raise your hand(s) Apr 15 12:03
* | viking_ice does power raise.. Apr 15 12:03
* | tk009 lurking Apr 15 12:04
* | f13 here Apr 15 12:04
jlaska | greetings folks Apr 15 12:04
jlaska | okay let's kick things off ... rough agenda posted to https://www.redhat.com/archives/fedora-test-list/2009-April/msg00978.html Apr 15 12:05
adamw | wow, that's a very emo nick. Apr 15 12:05
f13 | ? Apr 15 12:05
adamw | oh, sorry, thinking out loud. Apr 15 12:05
viking_ice | seconds f13: ? Apr 15 12:05
jlaska | I'll be posting minutes/tasks to https://fedoraproject.org/wiki/QA/Meetings/20090415 Apr 15 12:05
--- | jlaska has changed the topic to: Fedora Quality Assurance Meeting | Previous Meeting follow-up https://fedoraproject.org/wiki/QA/Meetings/20090408 Apr 15 12:07
jlaska | [jlaska+adamw] mediawiki semantic update (packaging and hosting) Apr 15 12:07
jlaska | I've ping'd out to the infrastructure folks for a snapshot of the fedoraproject.org/wiki to play with in a private mediawiki+semantic instance Apr 15 12:08
wwoods | are you reviewing *last* week' Apr 15 12:08
jlaska | yeah ... Apr 15 12:08
wwoods | er. last week's review? I'm all confused. ok sorry. Apr 15 12:08
* | wwoods not on his normal workstation, feels all out-of-sorts Apr 15 12:08
jlaska | no worries ... I carried fwd the tasks into todays agenda (https://fedoraproject.org/wiki/QA/Meetings/20090415#Previous_meeting_follow-up) Apr 15 12:09
jlaska | I suspect mediawiki+semantic will likely mature post F11 ... just too much to juggle to get that going at present Apr 15 12:09
jlaska | unless there are other ideas/suggestions, I'll likely keep tabs on that and pursue post-F11 Apr 15 12:09
* | jlaska moves to next item Apr 15 12:10
jlaska | # [jlaska] - reach out to pmuller on packaging rhtslib Apr 15 12:10
wwoods | yeah I think we're getting too close to F11 for a lot of our longterm development stuff. as always. Apr 15 12:10
jlaska | wwoods: true true :( I feel good we have some next steps lined up, but I suspect it'll have to wait Apr 15 12:11
jlaska | re: rhtslib ... I was hoping to get pmuller to join the meeting today as a "special guest" ... but was unable to get in touch with him Apr 15 12:11
jlaska | wwoods and I ping'd pmuller earlier this week, hopefully we'll have an update near th end of this week Apr 15 12:11
jlaska | wwoods: do you have any rhtslib packaging news? Apr 15 12:12
wwoods | jlaska: nothing to report Apr 15 12:13
jlaska | wwoods: okay ... we'll both follow-up later this week Apr 15 12:13
jlaska | # [jlaska] - improve the kickstart file used for generating test day live images Apr 15 12:14
jlaska | last week there was some list discussion around improving the kickstart file used for generating the test day live images Apr 15 12:14
jlaska | I spoke to jeremy about adding a kickstart script to the spin-kickstarts git repo ... jeremy suggested that might not be the best fit and instead suggested this: https://fedoraproject.org/wiki/QA/Test_Days/Live_Image Apr 15 12:15
jlaska | which is linked to from the main Test_Days page under the FAQ Apr 15 12:15
jlaska | it's nothing complicated, but I think meets the guidance supplied by adamw :) Apr 15 12:15
jlaska | "keep it simple" :P Apr 15 12:15
adamw | just reading Apr 15 12:15
adamw | nice Apr 15 12:16
jlaska | the kickstart sample posted still needs help in terms of creating a Test_Days:Current .desktop link and other suggestions from the list Apr 15 12:16
jlaska | but it's a start ... and anyone with tips/ideas is welcome to add on Apr 15 12:16
adamw | that looks like a very sensible way to do it Apr 15 12:16
adamw | maybe send a mail to the list about it? Apr 15 12:16
jlaska | adamw: ah doh! good idea, thank you Apr 15 12:17
jlaska | okay next up ... Apr 15 12:17
* | jlaska takes a TODO for mailing fedora-test-list@ Apr 15 12:17
jlaska | # [adamw] - send details of nss rawhide issue to warren for posting to rawhidewatch.wordpress.com Apr 15 12:18
f13 | afaik all the nss issues have been resolved as of last night Apr 15 12:18
f13 | there were odd arch issues, then there was a FIPS mode issue in installer, but all of that is resolved AFAIK Apr 15 12:18
adamw | and I did send the info, and it got posted Apr 15 12:18
--- | thomasj_ is now known as thomasj Apr 15 12:19
* | jlaska is behind on his planet reading :( Apr 15 12:19
adamw | f13: the main point of the note on rawhidewatch is for people who got stuck with the problem - it broke rpm, so you needed a special workaround to get out of it Apr 15 12:19
f13 | yeah, i'm about 3 months behind Apr 15 12:19
f13 | adamw: I didn't know that, I never hit the issue Apr 15 12:19
jlaska | http://rawhidewatch.wordpress.com/2009/04/08/rawhide-x86_64-firefox-and-rpm-broke-workaround-procedure/ Apr 15 12:19
wwoods | it was x86_64 only or something, right? Apr 15 12:19
adamw | yeah - basically it was x86-64 only and only happened if you happened to have packages installed in such a combination that the i586 package would get installed to satisfy a dep that should have been x86-64 only Apr 15 12:20
adamw | anyhow, it's all spelled out there so no need to go into details :) Apr 15 12:20
jlaska | adamw: thanks for following through, any other comments on that? Apr 15 12:20
adamw | nope, nothing springs to mind Apr 15 12:21
jlaska | okay Apr 15 12:21
adamw | just keep an eye out to do the same thing in future Apr 15 12:21
adamw | where there's an important rawhide issue that crops up on test-list or devel-list, just make sure we get a note into rhw Apr 15 12:21
wwoods | in the future we should talk about having a special set of bookmarks for rawhide - include rawhidewatch, test day listings, etc. Apr 15 12:22
wwoods | not just on the live images Apr 15 12:22
f13 | problem is that there can currently only be one bookmarks package Apr 15 12:22
f13 | we really need there to be multiple Apr 15 12:22
* | jlaska has a highlander flashback Apr 15 12:23
wwoods | well, we could just have a package that provides an extra menu full of tester-specific stuff Apr 15 12:23
wwoods | including web links and such Apr 15 12:23
jlaska | wwoods: not a bad idea if we can figure out a way to wedge it in Apr 15 12:24
jlaska | I've added that to open-discussion Apr 15 12:24
wwoods | it's a good idea, but as always: we don't need more good ideas, we need code Apr 15 12:25
jlaska | that does touch on the handy qa bookmarks for the test day live images topic too ... so I'm interested in figuring something out there Apr 15 12:25
jlaska | okay ... these next 2 topics related to what I was hoping we could spend some brain cycles on today Apr 15 12:25
jlaska | # [jlaska] - talk to poelcat about adding a few F11Blocker review meetings to the schedule Apr 15 12:25
jlaska | I spoke to poelcat and we do have blocker review days in the schedule (see item#39 in http://poelstra.fedorapeople.org/schedules/f-11/f-11-releng-tasks.html) Apr 15 12:26
jlaska | I'll come back to this shortly Apr 15 12:26
jlaska | adamw: you have a similar item ... you can update here or we can hold off for the general bug review part? Apr 15 12:26
jlaska | # [adamw] - review Test Day X11 bugs to ensure they are represented on F11Blocker Apr 15 12:26
jlaska | adamw: you're call? Apr 15 12:26
adamw | let's see - I was looking at nouveau bugs, then checked in with darktama and he says he's already doing a review of all of them himself Apr 15 12:27
adamw | so that should be covered Apr 15 12:27
adamw | radeon and intel I will talk to matej and francois about Apr 15 12:27
adamw | stick that down as an action item for me for next week? Apr 15 12:28
* | jlaska sticks Apr 15 12:28
adamw | i am a bit worried about radeon, there are rather a lot of open reports on it, several looking quite serious Apr 15 12:28
adamw | so i'll definitely want to go through those Apr 15 12:28
jlaska | adamw: okay I've noted that for next week ... along with radeon concerns Apr 15 12:29
jlaska | adamw: any other updates on your end? Apr 15 12:29
adamw | er, on what exactly? Apr 15 12:30
wwoods | i8xx still seems problematic Apr 15 12:30
jlaska | adamw: on the X11 bug front Apr 15 12:30
adamw | ah Apr 15 12:30
adamw | intel...i haven't looked at the list of intel bugs yet, but my perception is it's not as bad as radeon Apr 15 12:30
adamw | and nouveau is working out surprisingly well Apr 15 12:30
adamw | radeon seems to be in the worst shape Apr 15 12:30
* | jlaska feels the radeon pain Apr 15 12:31
wwoods | radeon is the oldest and arguably the most complex - both the variety of hardware supported by the driver and the complexity of the hardware itself Apr 15 12:31
wwoods | it's a toughie. Apr 15 12:31
wwoods | luckily I have no radeon hardware at all. ha ha ha! Apr 15 12:31
jlaska | wwoods: i8xx ... sorry I'm not familiar, is that the pulse/alsa bug tracking you've been working on? Apr 15 12:31
adamw | so yeah, i'll definitely get those reviewed along with matej and francois Apr 15 12:32
adamw | jlaska: he just means intel, i think Apr 15 12:32
jlaska | ah okay, sorry Apr 15 12:32
wwoods | jlaska: no, intel i8xx series (pre-i915) Apr 15 12:32
* | jlaska shows his X11 rustiness Apr 15 12:32
adamw | and yeah, as will says, those chipsets are the most problematic for intel Apr 15 12:32
adamw | as they had a somewhat different architecture to the newer ones Apr 15 12:32
jlaska | okay ... any other updates from last week? Apr 15 12:32
wwoods | see e.g. 490366 Apr 15 12:32
adamw | so they tend to be a bit buggier than the newer ones (as well as being severely limited in some areas) Apr 15 12:32
adamw | but yeah, i need to sit down and go through the intel and radeon open bug lists to have a proper view, so i'll do that for sure. Apr 15 12:34
jlaska | wwoods: I don't have a spot in the agenda to talk about your pulsa/alsa efforts, but I know you've been spending some time on that. Did you want to give an update there? Apr 15 12:34
jlaska | s/pulsa/pulse/ Apr 15 12:34
wwoods | er, sure Apr 15 12:35
wwoods | basically: pulseaudio has gotten a bad reputation because of broken ALSA drivers and misbehaving applications Apr 15 12:36
wwoods | most of the latter has been fixed, but a bunch of audio drivers were still broken - notably intel8x0 and intel-hda Apr 15 12:36
wwoods | which are amazingly common Apr 15 12:36
adamw | they're the most common audio chipsets by a rather wide margin Apr 15 12:36
adamw | basically all onboard audio from the last 3-4 years is 99% likely to be one or the other Apr 15 12:37
wwoods | http://pulseaudio.org/wiki/BrokenSoundDrivers Apr 15 12:37
* | jlaska ponders another smolt possibility Apr 15 12:37
wwoods | people blame pulseaudio a lot, "uninstall pulseaudio" is the common "workaround" Apr 15 12:37
wwoods | vitrolic blog posts, etc. etc. Apr 15 12:37
wwoods | also it happens that my workstation has one of these chipsets and I can't listen to music anymore. boo! Apr 15 12:38
wwoods | so I've spent a couple weeks digging hard into pulseaudio and alsa-lib and kernel sources to try to get to the root of the problem Apr 15 12:38
jlaska | heh, that's a motivator Apr 15 12:38
wwoods | bug 472339 Apr 15 12:38
buggbot | Bug https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=472339 medium, medium, ---, kernel-maint, ASSIGNED, snd-intel8x0: timing unstable (snd_pcm_avail() overflows, signals POLLOUT when it shouldn't) Apr 15 12:38
wwoods | and bug 485734 Apr 15 12:39
buggbot | Bug https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=485734 low, low, ---, kernel-maint, MODIFIED, intel-hda: snd_pcm_avail() overflows Apr 15 12:39
wwoods | are the main bugs - both on F11Blocker Apr 15 12:39
jlaska | nice ... not that there are bugs, but that you have helped narrow it down Apr 15 12:39
f13 | how confident are we that this will get "fixed" for F11? Apr 15 12:39
wwoods | to make a long story short - working with Lennart (PA author) and Jaroslav (kernel sound maintainer) we've determined Apr 15 12:39
wwoods | a) PulseAudio is *not* at fault Apr 15 12:39
wwoods | b) Intel sound hardware sometimes is flaky about reporting certain bits about its internal state Apr 15 12:40
adamw | f13: if it's only glitch-free mode at issue, we have a fairly bulletproof last line of defence: just disable it on the problematic hardware Apr 15 12:40
adamw | that at least can be done without too much trouble Apr 15 12:41
adamw | if proper fixes to the drivers can't be done in time Apr 15 12:41
wwoods | https://bugzilla.redhat.com/show_bug.cgi?id=485734#c16 (reference for a)) Apr 15 12:41
buggbot | Bug 485734: low, low, ---, kernel-maint, MODIFIED, intel-hda: snd_pcm_avail() overflows Apr 15 12:41
wwoods | anyway, a bunch of fixes landed in the upstream alsa kernel Apr 15 12:42
wwoods | as of kernel -70 it should be fixed for a majority of cases Apr 15 12:42
* | jlaska queues up a "test" Apr 15 12:42
wwoods | we're working on some of the remaining cases - intel8x0 where the clock speed is being detected wrong Apr 15 12:42
jlaska | wwoods: thanks for the update ... that's what I was going to ask ... any items you wanted to chase down for next week? Apr 15 12:43
wwoods | http://git.alsa-project.org/?p=alsa-kernel.git;a=summary Apr 15 12:43
wwoods | the patches starting after 2.6.30-rc1 are the result of this work Apr 15 12:43
wwoods | the first two are in our kernel -70, I'm testing the other 4 now Apr 15 12:43
* | jlaska notes the -70 kernel already passes the previous failure scenario (> 30sec of sound) Apr 15 12:43
wwoods | there are some positive reports in the bugs about kernel -70 Apr 15 12:44
wwoods | audio works better on my workstation than it has since F9 Apr 15 12:44
wwoods | really hoping we can finally put this "just uninstall pulseaudio" nonsense behind us Apr 15 12:44
wwoods | if it wasn't so late in the game I'd recommend a test day for intel sound hardware Apr 15 12:45
adamw | yeah, i was going to say that, but we just don't really have the time, i don't think :\ Apr 15 12:45
jlaska | yeah I'd love one ... if a slot frees up perhaps we can figure something out Apr 15 12:45
adamw | *checks schedule* Apr 15 12:45
* | jlaska hasn't updated 05-14 with iBus yet Apr 15 12:46
wwoods | there's enough people CCd on those bugs that I'm fairly confident it's being well tested Apr 15 12:46
adamw | there's a potential slot april 28th or 29th, pretty late though Apr 15 12:46
adamw | what do you guys think? worth doing then, or not? Apr 15 12:47
wwoods | nah, too late Apr 15 12:47
wwoods | the hardware is so common and the test case is so simple Apr 15 12:47
jlaska | wwoods: perhaps if we keep a brief check-in for the next few meetings ... sounds like this is something you're actively engaged in already Apr 15 12:48
wwoods | I think we can just ask that people try to, y'know, play some music and make sure it works Apr 15 12:48
adamw | ok, sounds good Apr 15 12:48
adamw | i'll maybe post a quick thread on fedoraforums to see who's having trouble there Apr 15 12:48
jlaska | that's a good idea Apr 15 12:49
--- | jlaska has changed the topic to: Fedora Quality Assurance Meeting | autoqa update Apr 15 12:49
adamw | stick it down as an action item for me so i don't forget? Apr 15 12:49
jlaska | wwoods: I've got autoqa as a rolling topic each week, but I know as we get close to GA finding cycles is tough Apr 15 12:50
jlaska | adamw: thx, got it Apr 15 12:50
jlaska | wwoods: do you think it's reasonable to have some basic email notification of existing tests to autoqa-results-list before GA? Apr 15 12:52
* | jlaska thinking of repoclosure or similar test Apr 15 12:52
f13 | the test itself needs a bit of work Apr 15 12:52
jlaska | f13: what sort of work? Apr 15 12:53
* | jlaska looks at https://fedorahosted.org/pipermail/autoqa-results/2009-April/000000.html Apr 15 12:54
jlaska | okay, we can come back to this if there are no updates ... I want to save time for F-11 Blocker discussion Apr 15 12:55
jlaska | I'll sync up with wwoods to see if the current 'next steps' are still accurate (https://fedoraproject.org/wiki/QA/Meetings/20090408#Autoqa_update) Apr 15 12:56
f13 | sorry Apr 15 12:56
f13 | the test itself "passes" even if there are broken deps Apr 15 12:56
f13 | and it has some interesting issues with how it's being ran with temp cache data, and arch settings Apr 15 12:56
wwoods | sorry, problems on my machine Apr 15 12:57
jlaska | no worries Apr 15 12:57
jlaska | I don't mind so much if the repoclosure test still needs massaging, I think just seeing that the "tests" are automatically running and sending output somewhere Apr 15 12:58
wwoods | f13: can you tell me more about the "interesting issues"? we can work on fixing that Apr 15 12:59
wwoods | returning a useful result is pretty easy to fix Apr 15 12:59
wwoods | but I'm not sure about the other stuff Apr 15 12:59
f13 | I think we need to make it use a temp cache dir for the repodata Apr 15 12:59
f13 | but I need to run it through again to see the output Apr 15 13:00
jlaska | wwoods: f13: are you guys up for meeting later this week to hash through any issues on autoqa? Apr 15 13:00
wwoods | okay Apr 15 13:00
wwoods | jlaska: yeah, that'd be great Apr 15 13:00
jlaska | I can setup a time to chat etc... Apr 15 13:00
f13 | yeah... it's going to be busy this week, but sure Apr 15 13:00
* | jlaska TODO's Apr 15 13:01
--- | jlaska has changed the topic to: Fedora Quality Assurance Meeting | F-11 Blocker Bugs Apr 15 13:01
jlaska | I was hoping to spend some time today discussing ways we can invite help from others for assessing blocker bug status Apr 15 13:02
jlaska | we've talked about this in the past, and it's tough issue Apr 15 13:02
jlaska | wwoods and I were discussing this week and we both mentioned that much of the decision about whether something shoudl be on the blocker list is a 'gut' feel based on experience Apr 15 13:02
adamw | i've been trying to ask bugzappers to flag up bugs in their particular components as blockers, but getting some resistance as they feel they might not do it right Apr 15 13:03
f13 | yay! new glibc building which should fix mysql (and emacs?) Apr 15 13:03
wwoods | expanding and clarifying releasecriteria (or having separate blockercriteria) might be a good idea Apr 15 13:03
jlaska | I can't help but feel we can describe a bit of that 'gut' feel in an effort to solicit help from others for assessing blocker bugs Apr 15 13:03
adamw | jlaska pointed to the wiki page to be used as a reference, but everyone felt it was a bit below scratch Apr 15 13:04
jlaska | wwoods: yeah you've got a great start alread on th release criteria page https://fedoraproject.org/wiki/QA/ReleaseCriteria Apr 15 13:04
wwoods | there's some fairly well-defined ways to evaluate stuff - severe bugs in Important Apps, any data corruptor in any package Apr 15 13:04
jlaska | adamw: cbeland has the priority/severity draft - http://fedoraproject.org/wiki/User:Beland/Bugzilla_Legend#Severity_and_Priority Apr 15 13:04
wwoods | severe bugs in firefox might be a blocker, while severe bugs in frozenbubble generally wouldn't Apr 15 13:04
wwoods | except IIRC that's not on the release criteria Apr 15 13:05
jlaska | are the QA release criteria and the evaluation of blocker bugs the same things? Apr 15 13:05
jlaska | I feel like one is definitely a QA decision/result Apr 15 13:05
adamw | wwoods: that's basically my take on 'priority' and 'severity', to rehash it for the tenth time :) Apr 15 13:06
jlaska | while the perhaps the blocker bugs should be more based on data (aka the severity listed by cbeland) Apr 15 13:06
jlaska | adamw: what's your take? Apr 15 13:06
adamw | just a sec while i read what beland wrote Apr 15 13:06
adamw | ah, he basically went with my system Apr 15 13:06
jlaska | here? http://fedoraproject.org/wiki/User:Beland/Bugzilla_Legend#Severity_and_Priority Apr 15 13:06
adamw | in which case, yes, priority is something of an overlap with blocker bugs Apr 15 13:07
adamw | but they're not really the same Apr 15 13:08
jlaska | adamw: what severity and priority? Apr 15 13:08
adamw | jlaska: on that page Apr 15 13:08
adamw | i mean, priority isn't the same as blocker evaluation Apr 15 13:08
jlaska | agreed Apr 15 13:08
adamw | first of all, priority is more fine-grained. second of all, it covers bugs in released versions as well as rawhide. Apr 15 13:08
adamw | so, uh. sorry. big topic. let me organize :) Apr 15 13:09
adamw | i guess if we adopted that system going forward, triage would naturally feed into blocker bug evaluatio Apr 15 13:09
adamw | n Apr 15 13:09
adamw | because we could simply look at all bugs with priority 'urgent' in rawhide as a step in evaluating blockers Apr 15 13:09
adamw | but that draft is still contingent on my poking the developers for feedback, which is on my todo list on the bugzappers side Apr 15 13:09
f13 | I'm still not convinced that exposing and actually trusting severity/priority in bugs is worthwhile. Apr 15 13:09
f13 | every person's pet bug is high priority high severity Apr 15 13:10
adamw | f13: i think we should at least try it, and if it turns out to be train wreck, we can stop again. but as i've mentioned, i've generally found that once it's been set by a triager, most reporters respect that Apr 15 13:10
f13 | if we didn't let the reporter frob that knob it would be different. Apr 15 13:10
adamw | yeah, that's certainly a possibility if it turns out to be a problem - but maybe it won't. Apr 15 13:10
f13 | adamw: well, it has been a train wreck in Red Hat land up to the point where most maintainers just flat out ignore it Apr 15 13:10
jlaska | I agree ... we need to invite others to help with this ... my concern is it's too much for a small group of folks to maintain Apr 15 13:10
adamw | f13: because it's being set by reporters Apr 15 13:11
adamw | who obviously can't do it right because we don't have any kind of policy or guidelines on what the hell they mean Apr 15 13:11
jlaska | and if we want help from others, we need to provide some breadcrums/guidance Apr 15 13:11
f13 | adamw: and when it was adjusted by the maintainer the reporter puts it back, and you get a revert war Apr 15 13:11
wwoods | the entire blocker-bug system is kind of a workaround for the fact that users can (and do) set priority capriciously Apr 15 13:11
f13 | so most of us are trained to simply ignore what the reporter puts it at all together and avoid the needless battle Apr 15 13:11
adamw | f13: as i said, that's just not been my experience - only a handful of reporters get into revision wars Apr 15 13:11
adamw | but anyhoo Apr 15 13:12
adamw | my proposal is, let's try it and see what happens Apr 15 13:12
f13 | perhaps mdv's users are different from Fedoras (: Apr 15 13:12
adamw | it can't hurt anything really Apr 15 13:12
adamw | it's not like we can wield a big stick and force maintainers to respect those settings Apr 15 13:12
adamw | if they want to ignore them, they will Apr 15 13:12
f13 | I'm all for trying, but don't expect maintainers to actually care about severity/priority settings Apr 15 13:12
wwoods | the whole system would be simpler if we just had a "release blocker" priority, but it doesn't work quite right Apr 15 13:12
adamw | i'm hoping that if it succeeds, they will *want* to, because the whole point is it helps maintainers Apr 15 13:12
f13 | what would you consider "success" in your trials? Apr 15 13:12
adamw | no revert wars, positive feedback from maintainers looking at their bug lists Apr 15 13:13
jlaska | community participation in the escalation of blocker bugs Apr 15 13:13
adamw | yes Apr 15 13:13
f13 | I often wonder how many maintainers actually look at their bug list directly, as opposed to filtered through something like F11Blocker et al Apr 15 13:13
jlaska | there are going to be some pain points I'm sure Apr 15 13:13
adamw | if we *do* get problems with revert wars, then we look at how to fix that, if we can't, we dump the system and just hide the damn fields Apr 15 13:13
f13 | (or bugs with certain flags on them internally) Apr 15 13:13
adamw | f13: i'd be distinctly worried about any maintainer who didn't care about any bug in his package that wasn't listed as a blocker. that really shouldn't happen. Apr 15 13:14
f13 | I would suggest you come up with the plan of how to use them, and what measurements you will use to determine success/failure and pitch it to FESCo as an F12 experiment Apr 15 13:14
adamw | if you're maintaining a package you should be able to be vaguely on top of the bug list for it. even if on top of means closing trivial bugs with a comment saying you don't have time to fix 'em. Apr 15 13:14
jlaska | f13: I'm looking for a solution to help manage the F11Blocker list Apr 15 13:14
f13 | adamw: when you've got a bug list that is hundreds long, you only have time to worry about the things that are blockers Apr 15 13:14
viking_ice | joins late in the game no point using or changing priority on report I thought everybody knew that maintainers dont give a frack about the severity/priority on reports? Apr 15 13:15
f13 | closing trivial bugs isn't the right thing to do, closing it doesn't make the bug go away Apr 15 13:15
adamw | viking_ice: the reason they don't is that it's never set usefully Apr 15 13:15
adamw | viking_ice: the point of the enterprise is to have them set usefully so it will actually be helpful to maintainers to care about them Apr 15 13:15
f13 | jlaska: define "manage" Apr 15 13:15
wwoods | it's a cycle. the fields are ignored because they're useless, and they're useless because they're ignored Apr 15 13:15
jlaska | that's a good point ... at this point I'm not so interested on whether these values serve as a priority for development Apr 15 13:15
jlaska | to me, that's later down the road Apr 15 13:15
viking_ice | adamw: wrong the reason is they them self are the only once that can judge where on their priority list the bug is Apr 15 13:16
viking_ice | s/once/one Apr 15 13:16
adamw | remember, I've *been* a package maintainer, and I was a lot happier when I could work on my bug list from critical down to trivial rather than having to try and figure it out myself or just take them in numerical order Apr 15 13:16
viking_ice | Then you must have been the only one.. Apr 15 13:16
jlaska | f13: I'd like to see that all the items on the blocker list pass some agreed criteria for being on there Apr 15 13:16
adamw | viking_ice: not in my experience. i've been in the position of maintaining several dozen packages at once and i found the judgment of experienced triagers very useful, if i'd spent all my time trying to prioritise the bugs i'd never have had time to fix any of them. Apr 15 13:16
f13 | the nature of Fedora right now is that there are far more bugs than people to work on them, regardless of priority/severity Apr 15 13:16
jlaska | f13: agreed ... and that's okay Apr 15 13:17
f13 | and Fedora is a serve yourself world, where maintainers of the packages get to choose what they want to work on Apr 15 13:17
adamw | sure. i'm not planning anything to try that Apr 15 13:17
adamw | s/try/change/ Apr 15 13:17
f13 | some groups, such as QA and releng, can offer suggestions of what is more important to fix, with the various trackers/blockers Apr 15 13:17
adamw | just provide more useful input Apr 15 13:17
viking_ice | adamw: being both the maintainer and the developer on the application? Apr 15 13:17
adamw | there's a logical problem here: the thrust of your arguments appears to be "there's no point bothering about bug priority because maintainers will only work on what they want to", in which case, why do we have a blocker list? Apr 15 13:18
jlaska | f13: that's a good point. Getting something off the blocker list doesn't mean it has to be fixed ... it can be release noted, or just ignored (which hurts everyone) Apr 15 13:18
f13 | adamw: as long as you pitch it that way I think you'll find much less push back Apr 15 13:18
f13 | adamw: always gear it towards "informative" rather than declarative Apr 15 13:18
adamw | f13: that's what i've been saying all along Apr 15 13:18
adamw | i'm not sure why you think i'm not Apr 15 13:18
f13 | adamw: the first instinct of many people will be to think that QA will want to force what people will work on Apr 15 13:18
adamw | i already *said* there's no way we can force anyone to care about the fields, all we can do is provide them Apr 15 13:18
f13 | mostly because QA/PM does that in RHEL Apr 15 13:19
adamw | well, that's not the idea at all. Apr 15 13:19
jlaska | f13: that's not right entirely there Apr 15 13:19
jlaska | f13: we just provide the data Apr 15 13:19
f13 | not only force what you should be working on but prevent working on anything else Apr 15 13:19
f13 | jlaska: qa ack? Apr 15 13:19
f13 | without it nothing can be worked on Apr 15 13:19
jlaska | f13: there's 3 parts to that ack Apr 15 13:19
f13 | sure Apr 15 13:19
jlaska | and ther'es the schedule Apr 15 13:19
jlaska | so qa ack is no guarruntee Apr 15 13:19
jlaska | it's just a way of getting it on the Blocker list Apr 15 13:19
jlaska | much the same way we do now Apr 15 13:20
f13 | but at the end of the day, people besides the developer force what is worked on Apr 15 13:20
jlaska | and "fixing" could mean code, documentation / release note Apr 15 13:20
f13 | in RHEL Apr 15 13:20
f13 | that's going to be the immediate assumption/fear around anything remotely close to that in Fedora land Apr 15 13:20
f13 | its a touchy subject Apr 15 13:20
jlaska | definitely Apr 15 13:20
jlaska | so certainly something we need to be careful in messaging Apr 15 13:20
adamw | anyhow, we're spinning wheels - point is, we're working on this, it's advisory, if it works great if it doesn't we'll kick it to the curb, and we can evaluate down the road how severity/priority and blocker lists are/should be interacting Apr 15 13:20
f13 | I've always tried very hard to let the maintainers do what they think best in Fedora. Triage and setting of priority/severity can certainly help the maintainer make that decision, so long as they have the ultimate say Apr 15 13:21
adamw | i'll make sure to make that very clear in the proposal, then. i'm going to send a draft to test-list of the mail i will eventually send to devel-list, so you'll be able to see it and provide feedback at that stage Apr 15 13:21
f13 | cool Apr 15 13:21
adamw | one concrete thing that occurs is we can make it a rule for triagers not to touch priority / severity without the maintainer's permission *if the maintainer already touched it first* Apr 15 13:22
adamw | that will cover the case of maintainers who prefer to actively manage those settings themselves Apr 15 13:22
jlaska | and we can always reach out to solicit a maintainer group to join a "pilot" Apr 15 13:23
adamw | yes - well, basically it would be a group to provide active feedback Apr 15 13:23
adamw | i.e. see if there are people who are willing to take a periodic look at their bug list ordered by severity/priority and see if it's useful to them Apr 15 13:24
viking_ice | this will never work.. Apr 15 13:24
adamw | oh, quit being pessimistic :) Apr 15 13:24
jlaska | adamw: do you have any thoughts on how this can be used to guide blocker bug input? Apr 15 13:24
adamw | i mentioned that way up above :) basically i'd say the obvious thing initially would be to use a search on 'urgent' priority bugs in rawhide as an input to the blocker list Apr 15 13:25
jlaska | sorry yes you did Apr 15 13:25
jlaska | yeah that seems sensible (to me) Apr 15 13:26
adamw | ok Apr 15 13:26
adamw | still, it depends on the system working out, and triage covering a sufficient mass of bugs, so we'll have to see how those pan out Apr 15 13:26
jlaska | I suspect we're not going to be able to build a ruleset that applies to 100% of the bugs Apr 15 13:26
f13 | priming the blocker list will never be a single data point Apr 15 13:26
adamw | no, but that's generally true for triage - it's very hard to write out a prescriptive list of everything Apr 15 13:27
jlaska | but if we can have something that helps manage the 80% or higher ... it's worth trying imo Apr 15 13:27
adamw | i think the guidelines on that draft page are a good start, and further to that we can take things as they arise Apr 15 13:27
f13 | whatever helps us or maintainers find the critical issues sooner rather than later will help Apr 15 13:27
adamw | right Apr 15 13:27
jlaska | exactly Apr 15 13:27
adamw | so, this is a medium-term thing: to go back to the initial discussion, in the short term, how do we broaden input into the blocker lists without losing signal? Apr 15 13:28
adamw | to take a concrete example, do we want bugzappers to contribute to the list, and if so how can we help them do it productively? Apr 15 13:28
wwoods | We almost never drop anything from Target Apr 15 13:29
wwoods | we can encourage people to 'nominate' bugs by adding them to Target Apr 15 13:29
wwoods | and then cherry-pick from Target to Blocker Apr 15 13:29
adamw | do we have a feel for how effective Target is, though? i kinda get the feeling it may be a bit of a dumping ground Apr 15 13:29
f13 | wwoods: I don't know if that will scale Apr 15 13:29
wwoods | It is indeed a bit of a dumping ground Apr 15 13:29
f13 | wwoods: often Target is too huge to get through for any one person Apr 15 13:29
f13 | my thought has always been that Target is useful for the individual maintainer who's bug gets put on it Apr 15 13:30
adamw | we could have a meeting, split it up into chunks, give each person a chunk to go through and flag up candidates for group discussion, i guess... Apr 15 13:30
f13 | but Target isn't wholly managed by any group or entity Apr 15 13:30
jlaska | hey guys, I'm sorry, I have a hard stop here Apr 15 13:30
f13 | Blocker gets much more attention and pruning Apr 15 13:30
adamw | f13: do you get a feeling that bugs added to Target get fixed with more urgency? i.e. maintainers find that status useful? Apr 15 13:30
wwoods | It could be managed by the bugzappers? Apr 15 13:30
f13 | adamw: I'd like to think so, but I have no evidence one way or another Apr 15 13:30
jlaska | adamw: f13: wwoods: you guys okay with continuing in #fedora-qa  ? Apr 15 13:30
adamw | yeah sure Apr 15 13:31
f13 | adamw: it'd be a good question to ask our maintainers if we're considering changing things. Apr 15 13:31
f13 | sure Apr 15 13:31
adamw | ok, noted Apr 15 13:31
viking_ice | it all boils down to developers work load hence he needs to set the severity/priority based on the time he has to work on the report ( which most of them user their own organizing system ) so reporters or triagers changing or setting the severity/priority will have no effect Apr 15 13:31
jlaska | thanks folks, sorry we didn't have time to open up for discussion ... please send mail to the list or continue in #fedora-qa Apr 15 13:31
--- | jlaska has changed the topic to: Channel is used by various Fedora groups and committees for their regular meetings | Note that meetings often get logged | For questions about using Fedora please ask in #fedora | See http://fedoraproject.org/wiki/Meeting_channel for meeting schedule Apr 15 13:32
jlaska | I'll follow up to the list with minutes ... thanks everyone :) Apr 15 13:32

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!

Retrieved from "https://fedoraproject.org/w/index.php?title=QA/Meetings/20090415&oldid=97203"