QA/Meetings/20130701

Attendees

• adamw (143)
• dan408\ (77)
• nirik (35)
• jreznik (32)
• tflink (28)
• cwickert (14)
• Southern_Gentlem (5)
• brunowolff (5)
• zodbot (4)
• Martix_ (4)
• dgilmore (4)
• robatino (3)
• Cerlyn (2)
• mkrizek (1)
• pschindl (1)

Agenda

• Fedora 19 retrospective and wrap-up
• Fedora 20 planning
• Taskbot
• Test Days
• Open floor

Fedora 19 retrospective and wrap-up

• The Fedora_19_QA_Retrospective page is now up and ready for feedback
• No-one had any other major post-F19 topics that weren't already in hand

Fedora 20 planning

• nirik and dgilmore will be proposing a new policy for deciding which spins to ship for F20
• adamw will continue with Final criteria rewrite and look at updating the validation test case set
• With the new Changes policy, the F20 'feature list' is Releases/20/ChangeSet
• jreznik would like to see functional validation of Changes for F20

Taskbot

• The current AutoQA setup is running on F17 which will go EOL soon: we may need to spend time moving it to something newer
• tflink says we're hoping to have a functional taskbot in place during F20 cycle

Test Days

• Special out-of-cycle request
• Deferred till next week

Open floor

• The spin-kickstarts package build process has been improved, which may make it less of a pain to do package updates

Action items

• adamw to add an item for Change validation discussion to next week's agenda

IRC Log

adamw	#startmeeting Fedora QA meeting	15:01
zodbot	Meeting started Mon Jul 1 15:01:54 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.	15:01
zodbot	Useful Commands: #action #agreed #halp #info #idea #link #topic.	15:01
adamw	#meetingname fedora-qa	15:02
zodbot	The meeting name has been set to 'fedora-qa'	15:02
adamw	#topic Roll call	15:02
* pschindl is here		15:02
* tflink is here		15:02
dan408\	hi	15:02
* mkrizek is here		15:02
* Cerlyn is here		15:03
adamw	set phasers to FUN	15:03
* jreznik is there, would like to leave a bit earlier, so go on, go on!		15:03
* brunowolff is here		15:03
adamw	#topic Fedora 19 retrospective and wrap-up	15:04
adamw	so, we got F19 tested and out the door with only a one week slip - big congratulations to everyone!	15:04
adamw	in case anyone missed the ML message:	15:05
jreznik	yep, thanks everyone!	15:05
adamw	#info I put the retrospective page up recently at https://fedoraproject.org/wiki/Fedora_19_QA_Retrospective	15:05
adamw	you can add feedback on the *QA* process to that page	15:05
Southern_Gentlem	adamw, my question is how the sudo switch is missing from the gnome install	15:05
adamw	Southern_Gentlem: the GNOME team does not want it to be optional.	15:06
adamw	it's by design for that tool.	15:06
adamw	that doesn't quite tie in with anaconda views it, but then i don't think we're quite at the point where GNOME gets to define the UI of *the entire distribution* :)	15:06
Southern_Gentlem	so first boot you have to give admin priviledges to the first user	15:06
adamw	if GNOME was designing the installer, it wouldn't have a switch either: the first user account would always be an admin.	15:06
Southern_Gentlem	something wrong with that	15:07
adamw	if you install GNOME and don't create a user in anaconda, yes.	15:07
* dan408\ sips coffee		15:07
adamw	so let's see, as well as general f19 stuff, some leading questions...	15:07
adamw	anyone particularly worried about any specific bugs that slipped through the net and have ideas to do anything about them?	15:08
tflink	other than the mac bug?	15:08
dan408\	no	15:09
tflink	nvm, already on the list	15:09
adamw	including that, if we have anything productive to contribute	15:09
adamw	though personally i'm not hugely concerned about that one	15:09
dan408\	it's good ol' fedora	15:09
tflink	yeah, I don't have anything to contribute that hasn't already been said	15:10
adamw	can anyone think of any issues we need to document that aren't documented already?	15:10
tflink	I'm minorly concerned about what criteria the cloud folks want to add, but that can wait until they're proposed	15:11
robatino	i'm already seeing bugs 946964/955779 on one of my newly installed F19 boxes. i think it may be a common issue	15:11
dan408\	nothing really, same usual questions popping up in #fedora	15:11
dan408\	grub questions, video drivers, getting up and running	15:12
dan408\	these are pretty much documented	15:12
adamw	cool	15:12
adamw	so no big new stuff	15:12
dan408\	no.	15:13
dan408\	i really dont see anything major at the moment	15:13
adamw	robatino: if you can clarify that one it could certainly be a commonbugs candidate, but I haven't had issues using gdm in KVM virt machines	15:13
adamw	thanks dan	15:13
dan408\	np	15:14
robatino	i'm seeing it on bare metal now (on a machine with blacklisted 82865 video)	15:14
robatino	i'll be investigating more	15:14
adamw	for the record, i've done most of the obvious commonbugs entries, the ones remaining are mostly ones that aren't entirely clear, but i'll take another pass through the list today	15:15
adamw	anything more for f19 or shall we move on to the next one? :)	15:15
jreznik	thanks adamw for commonbugs	15:15
dan408\	the next one ?	15:15
* dan408\ gets scared		15:15
adamw	f20!	15:16
dan408\	NO	15:16
dan408\	:(	15:16
dan408\	2 weeks vacation for all	15:16
adamw	#info Fedora 20 is cancelled, dan408's orders	15:16
dan408\	not cancelled	15:16
adamw	the official QA dictionary does not know the meaning of the word 'vacation'	15:17
brunowolff	I added a note about getting a TC for the minor spins.	15:17
dan408\	alright	15:18
adamw	brunowolff: that's really up to releng	15:18
adamw	we don't order them what to build	15:18
adamw	we just ask for a tC	15:18
jreznik	adamw: vacation? is it somehow related to vaca in spanish? :D	15:18
adamw	#topic Fedora 20 planning	15:19
brunowolff	It affected QA of those spins.	15:19
adamw	so F19 is SO last week	15:19
adamw	brunowolff: well, true.	15:19
* nirik would like to clearly propose dgilmore's spins critera for the f20 cycle.		15:19
adamw	nirik: can you expand on what you mean by that?	15:19
nirik	for tc/rc's we compose all approved spins, but spins must get 2 testers to ack that they passed basic tests. Those that do, are released for that milestone. Those that don't aren't.	15:20
jreznik	for f20 schedule reference - https://fedoraproject.org/wiki/Releases/20/Schedule (see, no earlier than)	15:20
nirik	if something doesn't get any acks for a cycle, it's dropped.	15:20
dan408\	can these acks be done via email?	15:21
* nirik was going to write up something for devel/test/spins		15:21
adamw	dan408\: good question	15:21
nirik	whatever folks would prefer, sure.	15:21
jreznik	nirik: well, we planned it in milano with cwickert too (similar, even for the big spins) - so I'm +1 for that	15:21
nirik	but a wiki might be easier to track	15:21
dan408\	+1 to a wiki on that	15:21
adamw	we have matrices for desktop spins, we don't print a matrix for every other spin	15:21
jreznik	nirik: or real ticket...	15:21
nirik	yeah, could get confusing tho... with all the spins...	15:22
jreznik	or matrix...	15:22
adamw	a page that just had 'does it boot?' for every spin would be easy to write, i guess.	15:22
* nirik doesn't care how we track it really.		15:22
adamw	does anyone see a major problem with the policy in general?	15:22
adamw	from a QA perspective? doesn't seem like a problem for us, to me	15:22
nirik	I was thinking: boots, can login, network works? (or any other basic things)	15:22
dan408\	as long as we aren't dropping spins left and right due to lack of acks, im +1	15:22
nirik	well, I suspect we will, but we will see.	15:23
adamw	i suspect the *two* tester requirement may be too many	15:23
adamw	we don't get two testers for KDE sometimes, enver mind anything else	15:23
nirik	should it be 1/	15:23
nirik	?	15:23
adamw	1 seems more realistic	15:23
cwickert	hold on	15:23
cwickert	It depends on what you want to test	15:23
adamw	and for a really basic functionality check, i don't see the need for double testing	15:24
nirik	anyhow, I am not saying we approve anything. I am just saying I intend to start a clear dialog on this for f20. ;)	15:24
adamw	cwickert: "<nirik> I was thinking: boots, can login, network works? (or any other basic things)"	15:24
adamw	nirik: sure, understood	15:24
dgilmore	adamw: my thought was the spin maintainer and one other	15:24
cwickert	most spins are based on other spins, so they should have pretty good coverage already	15:24
adamw	#info nirik and dgilmore will be proposing a new policy for deciding which spins to ship for F20	15:24
cwickert	like the jam-kde spin inherits testing from KDE	15:25
adamw	cwickert: the problem is that just adding packages can bust stuff sometimes, we can't assume a pass for the KDE spin means a pass for all KDE-derived spins	15:25
jreznik	dgilmore: spin maintainer + one other makes sense for me	15:25
nirik	except when it doesn't compose or has other issues.	15:25
adamw	jreznik: i think sometimes, 'spin maintainer' may be all we can get.	15:25
cwickert	all we then need to test is: 1) it composes, 2) it boots, 3) one can login and 4) all applications run as expected	15:25
dgilmore	adamw: i know in the past they have not tested	15:25
adamw	'all applications run as expected' is a large bear trap	15:25
cwickert	and this should IHNO be done by two people	15:25
cwickert	owner and one more	15:25
adamw	it takes about three hours to run that test for KDE	15:25
dan408\	+1 cwickert	15:26
adamw	and it's rarely a 100% pass	15:26
nirik	even spin owner is better than what we have now.	15:26
cwickert	s/IHNO/IHMO	15:26
dan408\	it takes 3 hours to run that test for kde?	15:26
adamw	yup	15:26
dan408\	is kde really that slow?	15:26
nirik	yeah, all applications is a bit crazy...	15:26
adamw	just launching every damn app in the menus and testing it can do _something_	15:26
dan408\	or are you talking about KDE + derivative spins?	15:26
cwickert	adamw: I don't expect it to pass 100, I just expect it to be tested	15:26
adamw	dan408\: no, it just has a lot of apps	15:26
jreznik	dan408\: manually, yes - there are a lot of apps, you have to try to click over, load document	15:26
adamw	dan408\: Desktop has rather fewer, still takes 1-2hrs though	15:26
Cerlyn	testing all apps on something like the security spin would be difficult	15:26
cwickert	adamw: why would it take 3 hours to test kde?	15:26
dan408\	jreznik: that's what we have you for	15:27
jreznik	definitely time to talk to vhumba about this one	15:27
nirik	I'd be fine with that as an optional.	15:27
cwickert	adamw: you mean according to the criteria I just outlined?	15:27
dan408\	well KDE is a "release blocking" desktop	15:27
adamw	cwickert: "all applications run as expected"	15:27
adamw	anyhow	15:27
adamw	we don't need to get bogged down in the details	15:27
dan408\	but for a basic spin you dont need to "launch every app"	15:27
adamw	yeah, that's kinda what I was saying, i don't think that test is even required	15:27
adamw	maybe check the key apps for the spin, sure	15:27
dan408\	yes.	15:28
dgilmore	so long as people can get in, get online and open a browser is a good start	15:28
dan408\	i.e. can you open a "konsole" or a "terminal"	15:28
dgilmore	that the apps work as expected is a plus	15:28
* nirik is with dgilmore. We should walk before we run. ;)		15:28
dan408\	okay	15:28
dan408\	i think we are all talking about the same thing here	15:28
adamw	yeah, i think dan's right	15:29
adamw	no-one seems to think the idea is a screaming failure, so let's just wait for the formal proposal and we can bikeshed the details then	15:29
adamw	everyone get your preferred tin of paint ready	15:29
cwickert	adamw: so, how many applications are there in the menu? say 50, starting one takes 10 seconds. that makes it less than 10 minutes. you just fire them up, and see if they start, you don't test each and every menu option	15:29
brunowolff	There are other details regarding arch and whether the image is optical, dd'd to usb or lcti'd to usb that should be considered.	15:30
adamw	cwickert: if you literally just want to see if they run, it might take a bit less. if you want to test at least that you can do some basic operation without it crashing, much longer.	15:30
* nirik notes f19 xfce fails on several apps in that test right now. ;)		15:30
* cwickert knows		15:30
dan408\	heh	15:30
cwickert	LXDE has two fails	15:30
* adamw waves the 'next topic' hammer threateningly		15:31
dan408\	maybe there should be a recompose of the live spins	15:31
dan408\	but i think someone is waving a hammer so i should run	15:31
Southern_Gentlem	a reduction in live spins	15:31
Southern_Gentlem	desktops and provide ks for rest	15:32
adamw	Southern_Gentlem: i don't think there's anything to be gained by arguing about that in a QA meeting.	15:33
adamw	alright	15:33
adamw	so other than this proposal, what do we need to do for F20?	15:33
dan408\	uh	15:33
nirik	nothing, lets ship it! :)	15:33
adamw	i have a few things lined up: I need to continue revising the final criteria (which got stalled during final validation), and i'd like to go through the whole validation test case set	15:33
dan408\	how are we pracitcally dealing with "features"	15:33
* nirik runs		15:33
dan408\	nirik: you have to compose it first	15:33
nirik	there are no more features. There are now changes. ;)	15:34
adamw	#info adamw will continue with Final criteria rewrite and look at updating the validation test case set	15:34
dan408\	nirik: right	15:34
adamw	dan408\: is the question 'how do things change for QA with the new Changes process'?	15:34
adamw	if so, excellent question	15:34
dan408\	yes	15:34
dan408\	sure	15:34
adamw	nirik: can you help us with that?	15:35
* dan408\ is trying to help with a "change" for 20		15:35
nirik	I could try. ;) I don't think things change too much aside from that the 'standalone' ones vs system wide might help with what you need to test?	15:35
dan408\	i guess for me it was more of a better understanding, and now i guess it's more for adamw to write criteron for standalone vs systemwide changes?	15:36
adamw	i don't know if there's some kind of obvious criterion	15:37
adamw	and i'm not the only one who can write criteria :)	15:37
dan408\	there is	15:37
adamw	if you think we need one, draft one up	15:37
dan408\	so i guess there should be a decision of whether certain systemwide changes should be "release blocking" or not	15:37
adamw	the distinction between 'standalone' and 'systemwide' should give us some clues of what to test	15:37
dan408\	fesco is no longer voting on these things anymore	15:38
adamw	well, what would it mean for a Change to be 'release blocking'?	15:38
dan408\	they are just proposed	15:38
dan408\	adamw: Well, do we plan any more anconda "improvements" for 20?	15:38
dan408\	Gnome 3.10 would be a systemwide change wouldn't it?	15:38
* jreznik would be glad so see qa involved in change process - especially what we miss now is the final ON_QA validation step - especially for the bigger system wide ones...		15:39
dan408\	Well what fesco used to vote on, who votes on it now? QA?	15:39
adamw	as in voting on whether features are 'approved'?	15:39
jreznik	dan408\: FESCo still votes on both	15:39
* nirik gets back. what?		15:40
dan408\	thank god	15:40
adamw	nirik: who's right, jreznik or dan?	15:40
jreznik	just for systemwide - FESCo wants change by change, for selfcontained - in batch	15:40
nirik	right.	15:40
nirik	actually I thought we were just approving selfcontained unless someone asked for a vote.	15:40
nirik	but either way.	15:40
jreznik	if somebody is not ok with selfcontained being selfcontained - it could be raised to fesco to vote on as nirik says	15:41
* dan408\ thinks all systemwide changes should be approved by fesco		15:41
nirik	all of them go to devel list for comment.	15:41
adamw	i think that's clearly the case	15:41
nirik	all systemwide ones must be passed by fesco	15:41
adamw	okay	15:41
dan408\	then self contained can just go to devel list	15:41
adamw	so we don't need to try and take that function over or anything, dan. but thanks for raising it	15:41
dan408\	right because there's obviously confusion there	15:42
adamw	dan408\: i think everyone's clear that systemwide Changes are reviewed, though. which is the important thing	15:42
dan408\	so i guess once that's done	15:42
adamw	we should certainly read the threads on devel@ and point out concerns we have with Changes from a QA perspective, btw	15:42
dan408\	we can get to testing these things	15:42
adamw	i've been doing that, but it'd be great if others can too	15:42
dan408\	well jreznik needs to send out the list..	15:43
dan408\	i kind of made a copy for 20 out of panic	15:43
jreznik	adamw: yes, that's one important thing from qa perspective	15:43
adamw	dan408\: each systemwide Change should show up as its own thread on devel@	15:44
jreznik	adamw: and that other part is validation of at least of systemwide changes	15:44
dan408\	http://fedoraproject.org/wiki/Releases/20/FeatureList	15:44
jreznik	aka is it really implemented?	15:44
dan408\	there is nothing up there	15:44
adamw	well, that gets back to the question I asked Dan above	15:44
jreznik	dan408\: it's not used anymore	15:44
adamw	what does 'validation' of a feature mean exactly?	15:44
nirik	we should remove/fix that page	15:44
dan408\	jreznik: which is bad imo	15:44
adamw	that the feature works 100%? that it works at all? that it doesn't cause breakage in anything else? what?	15:44
dan408\	how do we track this now	15:44
nirik	there will be another page without the word "Feature" in it?	15:45
adamw	good question dan	15:45
jreznik	dan408\: it's going to be replaced by https://fedoraproject.org/wiki/Releases/20/ChangeSet - being, sorry, only two accepted changes and I have to change my script to generate right list	15:45
jreznik	nirik: yes	15:45
adamw	nirik: i'd expect the 'FeatureList' location to re-direct to the new list for at least a few releases	15:45
dan408\	why are we confusing the hell out of people?	15:45
adamw	i don't know about anyone else, but XX/FeatureList is in *my* muscle memory	15:45
jreznik	adamw: of course I can do it	15:45
nirik	and possibly not in wiki... up to jreznik	15:45
dan408\	adamw: +1	15:46
dan408\	that is why i basically copied the wiki page at the time	15:46
dan408\	http://fedoraproject.org/wiki/Releases/19/FeatureList	15:46
dan408\	this is beautiful	15:46
jreznik	adamw: the idea was to extend the list with more info - not only a list but a place with details - see the ChangeSet template... there should be docs, marketing and I'm more than happy to track QA status there too	15:46
adamw	redirect to Releases/20/ChangeSet would be fine.	15:46
jreznik	adamw: ok, I redirect from old policy and yeah, I'll do it for a few releases too	15:47
adamw	so my take on 'validating features' has always been that, well, we validate the release	15:47
adamw	features are a part of the release	15:47
jreznik	for FeatuesList/ChangeSet	15:47
adamw	if they break something important, our validation tests should catch that	15:47
jreznik	adamw: release is validated, does it mean some changes are really implemented? no	15:47
dan408\	I guess we will repropose E for f20 as a "self contained" feature	15:47
adamw	but i'm not sure if that philosophy is water-tight; we might want to reconsider	15:48
jreznik	dan408\: do it please and let me knwo	15:48
dan408\	let's make sure that whatever is listed under the features category is moved to the changes category	15:48
adamw	jreznik: so, you want us to test that the Changes all actually work? that's another bunch of work for us to do, i guess	15:48
dan408\	#link http://fedoraproject.org/wiki/Features/Enlightenment	15:48
jreznik	adamw: one thing is - we know release is ok, but we are never 100% sure if all changes are ok and you know, we do release announcements (and many times we realized hey, it's actually not there etc.)	15:49
adamw	#info jreznik would like to see functional validation of Changes for F20	15:49
adamw	volunteers welcome...	15:49
jreznik	adamw: that's the question if it's doable...	15:49
adamw	well, we can look at that	15:49
adamw	for now let's move on as time is getting short	15:49
tflink	yeah, that sounds like a lot of additional work	15:49
dan408\	jreznik: I guess you just need to check http://fedoraproject.org/wiki/Category:FeatureReadyForWrangler	15:50
jreznik	tflink: it is but it's something we miss now :(	15:50
adamw	#action adamw to add an item for Change validation discussion to next week's agenda	15:50
adamw	let's discuss it in more depth next week	15:50
jreznik	dan408\: I'm in touch with owners and we're migrating it from features to changes	15:50
adamw	#topic Taskbot	15:50
dan408\	jreznik: okay	15:50
adamw	wanted to check in on the current status of taskbot	15:51
adamw	#chair tflink	15:51
zodbot	Current chairs: adamw tflink	15:51
adamw	what's the news, flinkonian one?	15:51
tflink	not much to say, getting ramped up now that F19 is mostly over	15:51
tflink	I want to get something in place before F20 starts up	15:52
adamw	that'd be nice	15:52
tflink	but most of AutoQA is running on F17, so that may need fixing sooner than later	15:52
adamw	ah, yes	15:52
adamw	#info current AutoQA setup is running on F17 which will go EOL soon: we may need to spend time moving it to something newer	15:52
tflink	the fedora version doesn't really matter much for our current tests	15:53
tflink	but it seems like bad form to be running on an EOL version	15:53
dan408\	well time to upgrade to 19 or "rawhide" ?	15:53
tflink	yeah, it's just a lot of work	15:54
adamw	i'm not sure if it's worth wasting a lot of work for 'bad form'	15:54
adamw	if there's no actual security issue to running an EOL'ed Fedora for the test machines...	15:54
tflink	they're all behind a firewall	15:54
dan408\	do they have selinux enabled? :P	15:55
tflink	everything publicly accessible is running el6	15:55
tflink	dan408\: not sure I see how that matters	15:55
tflink	they don't even have public ips	15:55
adamw	just a joke, i think	15:55
dan408\	kind of a joke	15:55
adamw	so, i'd consider that question before spending a bunch of time on it	15:55
adamw	especially if there is a realistic road ahead to taskbot in a reasonably short timeframe and this would delay that	15:55
tflink	yeah, I was hoping to avoid upgrading if we can	15:55
adamw	maybe work with infra to see what they think	15:56
tflink	about the upgrade?	15:56
adamw	yeah	15:56
tflink	that's all us	15:56
adamw	if they can think of an actual practical reason we may need to do it	15:56
adamw	ah okay.,	15:56
tflink	but it would be worth asking if they can see an issue with the plan	15:56
adamw	right, more input always valuable	15:57
adamw	as for taskbot, what were you hoping to get in place?	15:57
tflink	I	15:57
tflink	I'd like to get something mostly equivalent to AutoQA soon	15:58
nirik	reinstalling thing should be pretty easy I would think... but happy to discuss details.	15:58
tflink	nirik: there are some complications in doing that which lead back to the mess that is the method we use for sysadmin	15:58
adamw	well, that'd sure be nice if we can do it	15:58
adamw	is there stuff that people can help out with? I see john dulaney has been trying to help with depcheck testing	15:59
nirik	tflink: yeah, lets talk in admin later and hash out a plan.	15:59
* tflink has another meeting @ the top of the hour		15:59
tflink	at the moment, not much - there are still way too many variables	15:59
adamw	okay	15:59
* nirik nods. Later is just fine.		15:59
adamw	we can move the test day topic to next week	16:00
tflink	ideas for tests, or even better - code	16:00
Martix_	adamw: I'm here	16:00
Martix_	adamw: but I'll be here also next week ;-)	16:00
adamw	#info we're hoping to have a functional taskbot in place during F20 cycle	16:00
adamw	Martix_: the date on the special request wasn't super soon, was it?	16:00
Martix_	adamw: what special request?	16:01
adamw	it isn't, okay.	16:01
adamw	let's do that next week	16:01
Martix_	ok :-)	16:01
adamw	#topic open floor	16:01
tflink	wait, I got the time wrong on the next meeting. oh well not much to say right now, anyways	16:01
adamw	anyone have anything important we didn't cover?	16:01
dan408\	i think we're good	16:02
jreznik	no blocker meeting follow up today? :)	16:02
adamw	f20 blockers, sure :)	16:02
* adamw found one yesterday		16:02
dan408\	ack	16:02
brunowolff	Semi related to QA is that spin-kickstarts has been changed to be easier to build.	16:03
adamw	awesome, thanks bruno	16:04
dan408\	yup saw your email on that	16:04
adamw	#info spin-kickstarts package build process has been improved, which may make it less of a pain to do s-k updates	16:04
adamw	anything else, folks?	16:06
* adamw attaches Quantum Fuse to Cat		16:06
tflink	that brings up bad images	16:06
adamw	=)	16:07
adamw	okay, thanks for coming out everyone	16:07
adamw	#endmeeting	16:07

Generated by irclog2html.py 2.12.1 by Marius Gedminas - find it at mg.pov.lt!