QA/Meetings/20130218

Attendees

• adamw (120)
• Viking-Ice (47)
• nirik (27)
• cwickert (25)
• tflink (21)
• kparal (12)
• satellit (6)
• zodbot (3)
• jskladan (1)
• mkrizek (1)
• robatino (1)
• pschindl (1)

Agenda

• Automatic blocker proposal
• Blocker review process
• Open floor

Automatic blocker proposal

• The proposal was generally approved of by the group
• It was agreed to add Andre's suggestions and more strongly emphasize other bugs cannot be made automatic blockers

Blocker review process

• Limiting meetings to 3 hours and separating blocker review from QA meetings on Mondays seemed to work well during F18 cycle
• viking-ice proposed using a dedicated channel for blocker review meetings instead of -qa or -bugzappers
  • tflink and kparal had concerns about having another channel to monitor, but adamw and viking pointed out this would mean not needing to use -bugzappers any more

Open floor

• viking-ice wondered whether the nightly images were worth the effort of building: nirik said that building them is not much effort, and adamw and others said they were aware of some testing being done on them
• viking-ice suggested we have a basic smoke test matrix that all spins must go through at least once before being promoted as part of a final release
  • cwickert said he had made a similar proposal before and supported the idea

Action items

• adamw to write a second draft (of the automatic blocker proposal) with andre's proposed changes and stronger explanation not to put 'grey area' bugs in the automatic blocker list
• adamw to draft up changes to the blocker bug meeting SOP for 3-hour hard limit, no-reviews-during-qa-meetings, and a dedicated channel for meetings, send to list for further discussion
• viking-ice to discuss the 'smoke test for spins' idea further with nirik and cwickert

IRC Log

adamw	#startmeeting Fedora QA meeting	16:00
zodbot	Meeting started Mon Feb 18 16:00:05 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.	16:00
zodbot	Useful Commands: #action #agreed #halp #info #idea #link #topic.	16:00
adamw	#meetingname fedora-qa	16:00
zodbot	The meeting name has been set to 'fedora-qa'	16:00
adamw	#topic roll call	16:00
adamw	morning folks, who's around?	16:00
* kparal waves		16:00
* satellit listening		16:00
* mkrizek is here		16:00
* nirik is lurking		16:01
* pschindl is here		16:01
* tflink is here		16:02
adamw	hey, the gang's all here, now- OH NO WATCH OUT FOR THAT METEOR DRIVEN BY A RAPTOR	16:02
kparal	jskladan will survive and free us from the raptor dictatorship	16:03
adamw	wow, tough crowd.	16:03
adamw	heh	16:03
* jskladan lurks		16:06
* adamw waves from under meteor		16:06
adamw	hum, we don't seem to have a viking-ice yet	16:06
adamw	tflink: do you know what it was he wanted to discuss about the review process?	16:07
kparal	oh no, we're one viking short	16:07
adamw	if no-one knows what it was he wanted to talk about, we'll skip that item	16:09
tflink	adamw: wasn't it on the agenda for last week	16:09
tflink	nvm	16:10
tflink	oh, it was about the changes we kept making in F18	16:10
adamw	oh, the "QA:TestCase" topic from 0128?	16:10
tflink	keeping a static IRC channel, capping meetings @ 3hrs etc.	16:10
adamw	oh, i see.	16:10
adamw	well, let's do the other topic first	16:11
adamw	#topic Automatic blocker proposal	16:11
adamw	seems like most of the feedback on the 'automatic blocker' idea is +ve, i'll adjust it to incorporate andre's suggestions, any other thoughts?	16:11
adamw	#info the proposal is https://lists.fedoraproject.org/pipermail/test/2013-February/113840.html	16:12
tflink	it seems like a good idea to me	16:12
tflink	but it will increase the average time that we spend on bugs in meetings though :)	16:12
kparal	tflink: will it?	16:12
tflink	since we won't have the really easy ones bringing the average down	16:12
kparal	ah	16:12
kparal	the _average_ time	16:12
kparal	yes, bad for statistics! nack!	16:12
kparal	:)	16:12
adamw	heh	16:13
adamw	lies, damn lies, and tflink statistics	16:13
robatino	will there be automatic freeze exceptions as well (for oversized non-blocking desktops, for example)?	16:13
adamw	robatino: i didn't reply to that mail yet, but it seems reasonable	16:13
adamw	i'll try and come up with a new draft soon; i might emphasize the rules a bit harder too	16:13
adamw	so no-one can claim they misread it and just start slapping acceptedblocker on everything they propose	16:14
tflink	yeah, hopefully this won't be abused	16:15
tflink	but we won't know until we try	16:15
adamw	ah, the viking's here	16:16
adamw	Viking-Ice: we're on the 'automatic blocker' proposal - any further thoughts on that?	16:16
Viking-Ice	nope I agree to it fully	16:16
Viking-Ice	( I needlessly worried a bit about that gray area )(	16:17
adamw	cool	16:18
adamw	ok, i'll send out a second draft with andre's suggestions soon then	16:18
adamw	#info group generally supports the automatic blocker proposal	16:18
adamw	#action adamw to write a second draft with andre's proposed changes and stronger explanation not to put 'grey area' bugs in the automatic blocker list	16:18
adamw	#topic Blocker review process	16:19
adamw	Viking-Ice: we held this one in case you showed up - so, you said you wanted some discussion about this?	16:19
Viking-Ice	nothing comes to mind at the moment	16:21
Viking-Ice	so I got nothing new to add atleast	16:21
* adamw checks log		16:22
adamw	Viking-Ice well what did we learn about the blocker bug meetings	16:22
adamw	Viking-Ice well we should set a fixed channel and keep with the 3hour max limit	16:22
adamw	tflink #info discussion around the blocker review process for F19 would be wise before we get into testing	16:22
adamw	okay, that's where we were coming from.	16:22
adamw	so i guess this is about whether we want to formalize any of the f18 changes to the blocker bug meeting process	16:23
Viking-Ice	I think the 3 hour limit turn out working well but still I doubt ( or let's say I hope ) that we wont be experiencing that again this release cycle	16:24
tflink	we can hope :)	16:24
adamw	we all hope so :)	16:24
adamw	but it does seem like a reasonable rule indeed	16:24
tflink	but I suspect that it's going to keep happening every once in a while until/if we redo the process	16:24
tflink	but that's not happening for F19	16:25
Viking-Ice	and perhaps we should introduce new channel dedicated just for this ( not qa as some people wanted and not bugzappers and not meeting )	16:25
Viking-Ice	I also think it's better not to do blocker bug meetings in the midst of qa meetings or atleast I think it's better to just end the qa meeting and move to another channel	16:26
adamw	we did that once in the last cycle and it worked out fine	16:26
adamw	so it's a decent idea	16:26
tflink	yeah, I don't have any objections	16:27
adamw	it does get a bit messy having to look in qa meeting logs for blocker review	16:27
adamw	i can draft up a few changes to the sop	16:27
tflink	not sure about the dedicated channel, though unless we re-purpose #fedora-bugzappers	16:27
adamw	eh	16:27
adamw	i don't mind it	16:27
Viking-Ice	the reason I personally favor moving/using qa channel is likely hood of more participation	16:27
adamw	not like channels cost anything	16:27
Viking-Ice	tflink, we cant kill bugzappers if we continue to use it	16:28
tflink	it's just one more channel to join and keep an eye on :)	16:28
nirik	channels actually do cost. ;)	16:28
adamw	i can see viking's argument that using -bugzappers is kinda weird	16:28
adamw	the only reason to use -bugzappers any more is for these meetings though	16:28
nirik	they cost in attention of people...	16:28
adamw	so the net cost of a new channel is 0, as if we used one, everyone could quit -bugzappers...	16:28
tflink	that'd work for me	16:28
tflink	I don't much care about what the channel is named	16:28
tflink	but using a dedicated channel does open some interesting possibilities with irc bots in the future	16:29
adamw	Viking-Ice: i think we convinced him ;)	16:29
Viking-Ice	everyone is familiar and usually on the qa channel ( devs/qa community members ) but we might be interrupted like happened that one time if we use it in the midst of the meeting	16:30
adamw	okay, so how about this, i'll draft sop changes for all the above ideas and we can kick it around further on list	16:30
adamw	yeah, that's the problem with using -qa	16:30
adamw	it's a pretty active channel	16:30
Viking-Ice	I assume we want as much activity on that channel	16:31
Viking-Ice	( which usually means more vibrant and active community )	16:32
* nirik is happy with another channel as long as we kill bugzappers. net 0 is good.		16:32
adamw	okay.	16:32
adamw	Viking-Ice: sure, we want -qa to be active, but as you said, it gets awkward if we're having a two-hour blocker meeting and someone shows up wanting to chat about something else.	16:32
tflink	yeah, agreed that #fedora-qa is not the right place for review meetings	16:33
Viking-Ice	just throw it on the test list new channel any suggestion for the name of that channel and or use the qa channel and see how the community reacts/wants it	16:33
adamw	sounds good.	16:33
tflink	it sounded like a good idea when it was first proposed but in reality, it caused more problems than it solved :-/	16:34
adamw	#action adamw to draft up changes to the blocker bug meeting SOP for 3-hour hard limit, no-reviews-during-qa-meetings, and a dedicated channel for meetings, send to list for further discussion	16:34
adamw	okay then	16:36
adamw	looks like that's all we had on the agenda, so...	16:36
adamw	#topic open floor	16:36
satellit	anymore koji builds to test f19?	16:36
adamw	koji builds?	16:37
satellit	lives to test	16:37
Viking-Ice	I've been wondering a bit about that do we really need iso files ?	16:37
Viking-Ice	( other then alpha beta final )	16:37
Viking-Ice	as in nightly's	16:37
nirik	satellit: I have been holding off doing them while the mass rebuild is running. Should resume tomorrow or so.	16:38
adamw	they're useful, sure.	16:38
Viking-Ice	aren't we usually using them only to test anaconda?	16:38
satellit	I rely on the .iso's for soas	16:38
adamw	in f18 cycle we didn't use them a lot as we were making TCs almost constantly	16:38
adamw	but in previous cycles they've gotten a decent amount of use. not that weird to ask someone to check something with a nightly.	16:38
kparal	also tflink's composes lowered our usage of nightlies	16:39
Viking-Ice	do we have download stats on the iso's	16:39
adamw	not sure koji tracks that...nirik?	16:40
adamw	kparal: we're aiming to do fewer smoke builds for f19, to save tflink all the work.	16:41
nirik	I don't know that it does off hand...	16:41
nirik	there's probibly http logs.	16:41
* nirik could look if you like.		16:41
Viking-Ice	I've briefly been touching/pondering the idea if we somehow can use Colin Walters OStree to our advantage ( https://live.gnome.org/OSTree )	16:41
adamw	i remember reading his blog post on it and thinking 'hmm, that's interesting', but i didn't really have any concrete ideas	16:42
nirik	so, httpd logs are kept for iso downloads. What info from there would you find useful?	16:43
adamw	i think viking was curious about how much the nightlies are downloaded?	16:43
nirik	4569 downloads in 2012-12	16:44
Viking-Ice	adamw, yeah that's where I'm at came across it looks interesting wondering if we can take some kind of advantage of it but nothing concrete yet	16:44
Viking-Ice	nirik, each release?	16:44
nirik	5117 in 2013-01	16:44
Viking-Ice	or total	16:44
nirik	total	16:44
nirik	any ".iso " download	16:44
nirik	nightlys are only kept for a week or so tho.	16:45
Viking-Ice	I'm just trying to asses the benefit of using it	16:45
Viking-Ice	vs overhead	16:45
adamw	the overhead's pretty tiny	16:45
adamw	i think it's just nirik firing a script	16:45
nirik	yep.	16:46
adamw	if it doesn't build, we don't try and fix it	16:46
nirik	and we keep wanting to automate it.	16:46
Viking-Ice	makes sense	16:46
nirik	it's useful also for spins folks to test if they ever do	16:46
satellit	+1	16:46
Viking-Ice	technically gnome users should be testing the gnome spin as well while we try to focus our energy on the core function	16:47
Viking-Ice	but yeah	16:47
Viking-Ice	how much testing did other then the *DE spin get	16:48
Viking-Ice	I think those might be getting little to no testing even from their maintainers	16:49
nirik	I don't think there's any reasonable way to quantify that. ;)	16:49
adamw	in f18 not a huge lot, for f15->f17 i tried to get decent amount of testing for the non-blocking spins	16:49
* nirik did in fact test the Xfce spin a number of times.		16:49
adamw	we at least made sure the whole desktop matrix was done once or twice at each milestone	16:49
adamw	for xfce and lxde	16:49
adamw	satellit tests sugar quite a lot	16:50
Viking-Ice	I'm not worried about the *DE spins ( and sugar ;) ) they all have active communities it's the other ones that concern me	16:50
satellit	I also do VirtualBox installs from spins to test yum installs of other DE's with sugar	16:50
adamw	outside of the desktops and sugar, hell if i know.	16:51
adamw	you may well be right that they don't get much of a look.	16:51
Viking-Ice	I'm wondering if we should not come up with a test matrix for those that the spin maintainers have to walk through and "pass" before release	16:51
* nirik nods. Suggested as much to the spins list a while back.		16:51
adamw	i don't mind the idea in theory, as it does kinda suck when we ship stuff that's utterly borked	16:51
adamw	even if it's a spin we explicitly don't support	16:52
adamw	maybe you two could get together and re-propose it to spins?	16:52
Viking-Ice	what's releng take on something like that	16:52
Viking-Ice	( anything we might handout at various events needs to be thoroughly tested )	16:53
nirik	the spins setup is disfunctional, but attempts to fix it haven't met with anything concrete.	16:53
nirik	cwickert would be the one to involve in those discussions	16:53
cwickert	?	16:54
nirik	cwickert: spins process... didn't go so well last cycle. ;(	16:54
cwickert	yes, I know	16:54
Viking-Ice	cwickert, to bring you up to speed qa/releng requesting test matrix spin has to pass before being released	16:55
cwickert	but I'm afraid it will become worse when we kill the spins	16:55
adamw	i don't think we hand out anything but the multi-install and multi-desktop	16:55
adamw	and the regular install / desktop of course	16:55
cwickert	Viking-Ice: that means what exactly?	16:55
nirik	I was thinking a 2 person checkoff of a test matrix for each spin we want to promote on spins.fedoraproject.org. The rest can exist, just in a corner of alt.	16:56
Viking-Ice	cwickert, test matrix spin maintainers have to walk through which ensures atleast no surprises	16:56
Viking-Ice	for their spins	16:56
adamw	this is the proposal	16:56
cwickert	Viking-Ice: argh	16:57
Viking-Ice	cwickert, the *de spins are not much worries since those have active community's it's the other spins	16:57
cwickert	Viking-Ice: please consider me as an idiot who doesn't have a clue what a "test matrix spin maintainer" is	16:57
adamw	cwickert: there should've been some punctuation or grammar in there :)	16:57
Viking-Ice	uhum yes	16:58
cwickert	I know what a test is, I know what a test matrix is, I know what a maintainer is	16:58
adamw	the idea is that there would be *a* test matrix (basically just a test plan) that spin maintainers have to run through to have their spin 'approved' or whatever for a release, just a very basic 'does it boot?' smoke test	16:58
nirik	I think we are talking about "it boots, selinux is enforcing and works, it lets you login, etc"	16:58
cwickert	but who is supposed to maintain the test matrix for a spin?	16:58
adamw	cwickert: it'd be a generic one i think	16:58
cwickert	can there be specific tests for a spin?	16:59
cwickert	who is to maintain them and so on	16:59
cwickert	there is tons of questions	16:59
adamw	i think this idea would just be a very basic generic 'smoke test'	16:59
adamw	spin-specific tests are possible but would be a different thing	16:59
Viking-Ice	qa would maintain the matrix I suppose we already have criteria for "core" the rest is just packages on top of that	16:59
adamw	we actually already have one such matrix for the security lab spin (though no-one ever runs it)	16:59
cwickert	ok, I'm sorry, I need to stop here, FAMSCo meeting	16:59
cwickert	but we DO need to talk about this	16:59
adamw	anyway, it seems like a decent idea	16:59
* nirik nods.		17:00
adamw	#action viking-ice to discuss the 'smoke test for spins' idea further with nirik and cwickert	17:00
cwickert	I know the spins went badly	17:00
nirik	yeah, I don't think anyone disagrees... just how we improve them. ;)	17:00
cwickert	but on the other hand I am very frustrated about getting little or no feedback from QA about my requests	17:00
adamw	sorry, which requests?	17:00
Viking-Ice	yeah I missed those to	17:01
cwickert	adamw: changelogs in the announcements for the differenc milestones, better browsability in the wiki, meaningful renaming of the tracker bugs	17:01
adamw	oh, those	17:01
cwickert	are you coming to devconf?	17:02
adamw	1) i talked to andre about that one and we edited the text of the announcements somewhat to make it clearer that the 'changelog' is in the trac ticket	17:02
adamw	2) unfortunately didn't get to that one yet	17:02
adamw	3) we did that	17:02
cwickert	I'd appreciate if we can discuss some things	17:02
* Viking-Ice still lost...		17:02
adamw	cwickert: nope	17:02
Viking-Ice	I will be there kparal as well	17:03
adamw	Viking-Ice: these are requests from some time back, i don't recall exactly what the forum was but i recall the discussion now	17:03
adamw	cwickert: right, you can talk to viking and kparal there (probably also jskladan)	17:03
Viking-Ice	perhaps this should end up in our trac instance	17:03
adamw	cwickert: did you miss the tracker bug renaming thing? cos that was a whole thing a few weeks back.	17:03
kparal	in a pub :)	17:03
adamw	i think it may well be there	17:03
Viking-Ice	kparal, with rotten shark bits ;)	17:03
adamw	i'm pretty sure i filed tickets at the time	17:03
kparal	Viking-Ice: that's not really a czech speciality	17:04
adamw	https://fedorahosted.org/fedora-qa/ticket/273 is the ticket for the 'browsability' thing	17:04
adamw	https://fedorahosted.org/fedora-qa/ticket/272 is for the 'changelog' thing...it's not that we didn't give you any feedback, really, but andre didn't entirely agree with the proposal...	17:05
Viking-Ice	307 for some of it?	17:05
adamw	that wasn't part of cwickert's request, no. but i do need to finish that up. sigh	17:05
adamw	so much stuff to do	17:05
adamw	anyhoo, we're a bit over time	17:06
Viking-Ice	kparal, I will be bringing a box of bits for people to try as requested ;)	17:06
Viking-Ice	yup	17:06
adamw	so let's wrap up	17:06
cwickert	adamw: 3) we did that?	17:06
adamw	cwickert: the tracker bug renaming,.	17:06
cwickert	adamw: I don't think so	17:06
cwickert	what are the names now?	17:06
adamw	cwickert: https://lists.fedoraproject.org/pipermail/test/2013-January/113405.html	17:07
kparal	Viking-Ice: let's hope it's not an attempt to wipe out Brno's Red Hat office :-)	17:07
cwickert	I am searching the wiki for 5 minutes now for the NTH have bugs :(	17:07
adamw	cwickert: they're called FreezeException now	17:07
adamw	cwickert: and they're always listed at https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers	17:07
adamw	which is linked from https://fedoraproject.org/wiki/Blocker_Bug_FAQ and https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process	17:08
* adamw sets fuse		17:09
* cwickert needs to bail out for the FAmSCo meeting		17:10
adamw	cwickert: let us know what you think about the new names	17:11
cwickert	adamw: I made a proposal, so I probably prefer what I proposed, right?	17:12
adamw	well i'd *hope* not everyone thinks that way :)	17:12
adamw	i had a proposal too, and so did tflink, but we both prefer the final scheme	17:12
cwickert	I mean, the new names are better than the old ones, but still I consider mine better :P	17:12
adamw	anyhoo	17:12
adamw	time to end this nightmare!	17:12
adamw	thanks for coming folks	17:12
adamw	#endmeeting	17:12

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!