QA/Meetings/20101206

Attendees

People present (lines said)

1. jlaska (153)
2. adamw (58)
3. kparal (26)
4. pjones (23)
5. nirik (13)
6. mdomsch2 (9)
7. mkrizek (6)
8. fenrus02 (3)
9. Viking-Ice (3)
10. jsmith (3)
11. dgilmore (3)
12. abadger1999 (1)

Unable to attend:

1. Rhe (hopefully sleeping)

Agenda

• Proposed meeting agenda
• MeetBot summary

Previous meeting follow-up

1. jlaska to prep F15 test day wiki and request ideas on test@
2. Nirik to start discussion on test list related to updates ideas (see http://lists.fedoraproject.org/pipermail/test/2010-November/095756.html)

Many ways to get involved in QA ...

Call for Test Days

Owner

jlaska

Summary

Call for ideas/owners for Fedora 15 QA/Test_Days

Monitor list of approved features for ideas

AdamW initiated GNOME3 discussion on desktop@ list

Next steps ...

Contribute ideas to Talk:QA/Fedora_15_test_days

Identify owners to organize and host test days

Python script communicating with bugzilla

Owner

Bruno

Tune existing python scripts to detect UNTESTED blocker bugs

Next steps ...

See fedora-qa ticket#89

Requirements review for Fedora test case management

Owner

rhe

Requirements review for Fedora test case management

Next steps ...

See fedora-qa ticket#152

Critical Path test case development

Owner

adamwill

Critical Path test case development

Next steps ...

See fedora-qa ticket#154

AutoQA Update

Owner

kparal

Summary

1. All autoqa config files are now copied from the server to the client when executing tests
2. Watchers and autoqa script now return non-zero return code when test scheduling fails. No more unnoticed failures
3. jskladan is in the process of rewriting current bodhi watcher into watching koji -pending tags
4. Anaconda and depcheck tests were removed from master branch. There are not complete yet, we will re-add them once they are finished
5. clumens has written a new anaconda_storage test, it's present in the clumens branch
6. mkrizek worked heavily on posting Bodhi comments. It's almost finished, expecting additional changes this week

Next steps ...

1. Wwoods posting blog article explaining the need for autoqa ticket#248
2. Jskladan rewriting bodhi watcher to work with depcheck (encountering depcheck issues)

Open discussion - <Your topic here>

mdomsch - FTBFS

Owner

mdomsch

Summary

my FTBFS report includes ~80 bugs that now BFS

I'd appreciate someone _else_ verifying the logs, then closing them

begin creating the list of FTBFS bugs that will cause packages to be blocked from F15 at Alpha

starting with those ~110 that are still open from F14, and warning people

Next steps...

Team didn't feel FTBFS issues were an appropriate fit for QA group

Bring FTBFS discussion to rel-eng or FESCO

pjones - grub 4kb bugs

Owner

pjones

Summary

https://fedoraproject.org/wiki/Features/FourkBSectorBooting

rawhide/f15 now have support for booting from drives with 4kB sectors (as opposed to 512B like normal disks have) for UEFI only

Next steps ...

Folks installing rawhide starting from last Friday (12-03), be on the lookout for boot related failures

AdamW - Critpath test plans

Owner

AdamW

Summary

https://fedorahosted.org/fedora-qa/ticket/154

Adam is working on the wiki structure/layout to begin creating critpath test cases and plans

Next steps ...

1. Experiment with wiki document structure
2. Identify how testers (and tools/bodhi) will locate tests/plans
3. Begin drafting specific critpath tests

IRC Transcript

jlaska	#startmeeting Fedora QA meeting	16:00
zodbot	Meeting started Mon Dec 6 16:00:23 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.	16:00
zodbot	Useful Commands: #action #agreed #halp #info #idea #link #topic.	16:00
jlaska	#meetingname fedora-qa	16:00
zodbot	The meeting name has been set to 'fedora-qa'	16:00
jlaska	#topic Gathering ...	16:00
* jsmith lurks		16:00
jlaska	jsmith: lurker!	16:02
jsmith	Aye... guilty as charged	16:02
* jsmith is juggling cats this morning		16:02
* mkrizek here		16:02
jlaska	mkrizek howdy	16:02
adamw	yo	16:02
* jlaska electronically waves to adamw		16:03
adamw	isn't that illegal in most states?	16:03
jlaska	and some territories	16:03
adamw	=)	16:04
* Viking-Ice here		16:04
jlaska	Anyone else around? I suspect kparal is lurking, Viking-Ice, robatino, wwoods, jskladan etc...	16:04
jlaska	ah there we go, Viking-Ice: hiya	16:04
Viking-Ice	Greetings	16:04
* kparal forgot to announce himself :)		16:04
jlaska	wait another minute or so here ... and then will get started	16:04
jlaska	kparal: no need, we sensed your presence :)	16:04
jlaska	mkrizek: are we causing you to miss another lecture?	16:05
mkrizek	btw jskladan says hi, he had to go to school, he's taking a final exam today	16:05
mkrizek	jlaska: yes:)	16:05
* fenrus02 waves		16:05
jlaska	fenrus02: howdy	16:06
jlaska	oh no ... good luck jskladan	16:06
pjones	jlaska: another item for open floor: pay attention for grub failures. I'm somewhat worried about effects of the 4kB sector patch I put in thursday.	16:06
jlaska	pjones: that's coming in rawhide/F15?	16:06
pjones	it's in rawhide now.	16:06
jlaska	pjones: I'll bug ya for more details once we get there if that's okay	16:07
pjones	alright, just ping me when you get there.	16:07
jlaska	k	16:07
jlaska	let's get movin'	16:07
jlaska	#topic Previous meeting follow-up	16:07
jlaska	#info jlaska to prep F15 test day wiki and request ideas on test@	16:08
jlaska	the wiki templates/categories have been created ... I think I got all the pages	16:08
jlaska	I started tossing ideas into hte discussion page at https://fedoraproject.org/wiki/Talk:QA/Fedora_15_test_days	16:08
adamw	cool	16:08
adamw	i'll fill that out, actually have quite a lot of ideas lined up already	16:08
jlaska	adamw: sweet ... I figured the GNOME3 ideas would expand based on your review	16:09
jlaska	adamw: do we want to keep using that talk page as a "whiteboard" for ideas?	16:09
jlaska	or do we want to move this stuff into TRAC tickets soon?	16:09
adamw	yeah, trac tickets is good	16:10
adamw	but a talk page is fine for quickly gathering the ideas	16:10
jlaska	okay ... anyone want to fire off something to test@ to get some more ideas going?	16:11
* jlaska needs to add dramsey's dual-boot event to the list		16:11
jlaska	alright ... next was just a check-in with nirik ...	16:11
jlaska	#info Nirik to start discussion on test list related to updates ideas (see http://lists.fedoraproject.org/pipermail/test/2010-November/095756.html)	16:11
jlaska	nirik: around?	16:12
nirik	just barely. ;)	16:12
jlaska	heh, know the feeling :)	16:12
jlaska	nirik: if you had a moment, just wanted to follow-up with you on the thread you started last week	16:12
nirik	ok.	16:12
jlaska	Are you happy with the amount of feedback received? Are there still gaps in your opinion?	16:12
nirik	I meant to do some replying/collecting of ideas from there...	16:12
nirik	it seemed like good info/feedback. More is always welcome.	16:13
jlaska	certainly	16:13
jlaska	I think we addressed all the big items on the list ... but shout if there are any areas not detailed enough	16:13
jlaska	alright ... main agenda time ...	16:14
jlaska	#topic Many ways to get involved in QA ...	16:14
adamw	hold up	16:14
adamw	...	16:14
adamw	d'oh	16:14
jlaska	adamw: waddya got?	16:14
jlaska	#topic Previous meeting follow-up	16:14
adamw	just wanted to note one thing: we're clearly *really* waiting on Bodhi v2.0 (Now With Added Unicorns)	16:14
adamw	the non-numeric feedback is needed for all sorts of stuff	16:15
nirik	yeah. Any eta on that?	16:15
adamw	anything we (possibly an RH 'we' there) could do to kick it along would help	16:15
adamw	nirik: no, we've asked luke a few times but never quite pinned him down to a date	16:15
nirik	ok.	16:15
jlaska	okay	16:16
jlaska	adamw: thanks for the extra note ... I'll move on since I don't think there is much we can tackle on that subject here	16:18
adamw	yeah, just wanted to highlight it	16:18
jlaska	#info A lot of critical features planned for Bodhi-2.0 release, but unclear on ETA	16:18
jlaska	#topic Many ways to get involved in QA ...	16:18
jlaska	okay, this topic is kind of silly ... but I just wanted to point out that all of the following activites are ways for people to get involved	16:19
jlaska	and I think we have plenty for everyone	16:19
jlaska	we already talked about ...	16:19
jlaska	#info Call for F15 Test Day ideas - https://fedoraproject.org/wiki/Talk:QA/Fedora_15_test_days	16:19
jlaska	In discussion from some of the F-14 fedora-qa ticket cleanup, bruno mentioned this one ...	16:19
jlaska	#info Tweak python script to provide list of UNTESTED blocker bugs - see https://fedorahosted.org/fedora-qa/ticket/89#comment:11	16:20
jlaska	Two bigger efforts that will need more discussion and ideas ...	16:20
jlaska	#info Requirements review for Fedora test case management - see http://fedorahosted.org/fedora-qa/ticket/152	16:21
jlaska	#info Critical Path test case development - http://fedorahosted.org/fedora-qa/ticket/154	16:21
adamw	yup!	16:21
jlaska	and of course ... the ever-present AutoQA project has no shortage of tasks and tickets for folks to help with :)	16:21
jlaska	anything I missed in the general PSA?	16:21
adamw	not that i can think of	16:22
jlaska	we have few bodies, and plenty of tasks	16:22
jlaska	which is much more exciting than plenty of bodies and no tasks	16:22
jlaska	so if anyone wants to get involved ... I think we can find something tailored to any interest level	16:23
jlaska	okay okay ... I'll quit the PSA :)	16:23
jlaska	kparal: you ready for the main event?	16:23
* kparal is already pre-typing the text :)		16:23
jlaska	#topic AutoQA update	16:24
kparal	Here we go. Updates from AutoQA for the last week:	16:24
jlaska	kparal: I've got last weeks 'next steps' listed on the meeting wiki page if that helps (https://fedoraproject.org/wiki/QA/Meetings/20101206#AutoQA_Update)	16:24
kparal	ah	16:24
kparal	I'll come to that :)	16:24
jlaska	okay	16:24
kparal	1. All autoqa config files are now copied from the server to the client when executing tests. That will allow us to use "private" config files like fas.conf (containing credentials necessary for posting comments into Bodhi). It will also allow us to use local config files (e.g. checked out from git) instead of the system-wide ones. But more about that in the future.	16:24
jlaska	#info All autoqa config files are now copied from the server to the client when executing tests.	16:25
kparal	2. Watchers and autoqa script now return non-zero return code when test scheduling fails. No more unnoticed failures.	16:25
kparal	3. jskladan is in the process of rewriting current bodhi watcher into watching koji -pending tags. During the development we have found some problems with current bodhi implementation, which persuaded us that the rewrite is necessary.	16:26
jlaska	kparal: ignore me ... I'm back-filling with meetbot notes	16:26
jlaska	#info Watchers and autoqa script now return non-zero return code when test scheduling fails. No more unnoticed failures.	16:26
jlaska	#info jskladan is in the process of rewriting current bodhi watcher into watching koji -pending tags.	16:26
kparal	if anyone has any comments, feel free to post them, don't hesitate :)	16:26
kparal	4. Anaconda and depcheck tests were removed from master branch. There are not complete yet, we will re-add them once they are finished.	16:27
jlaska	I have one ...	16:27
jlaska	rockin'! :)	16:27
kparal	jlaska: yes please?	16:27
kparal	ah, thanks :)	16:27
jlaska	kparal: you know me ... I save the more boring comments for the list :)	16:27
jlaska	#info Anaconda and depcheck tests were removed from master branch. There are not complete yet, we will re-add them once they are finished.	16:27
kparal	5. Fedora 12 is EOL, we have removed it from the watchlist	16:27
jlaska	good bye sweet F-12, you will be missed	16:28
* jlaska checks autoqa.fp.org repoinfo.conf file		16:28
kparal	6. clumens has written a new anaconda_storage test, it's present in the clumens branch. It is a very complex test, that should cover most of the installation test cases from our release validation matrix	16:28
kparal	kudos to him	16:28
jlaska	kparal: I just updated production to disable F-12 test scheduling	16:29
kparal	jlaska: thanks	16:29
jlaska	yeah, really nice work clumens! ... you can read more at -- http://www.bangmoney.org/serendipity/index.php?/archives/161-This-Week-in-Anaconda-7.html	16:29
kparal	after anaconda_storage is in master, we should have anaconda installation tests executed after each anaconda build. yay!	16:29
jlaska	#info clumens has written a new anaconda_storage test, it's present in the clumens branch.	16:29
jlaska	kparal: and some day ... after each git commit :)	16:30
kparal	almost scary :)	16:30
jlaska	hehe	16:30
jlaska	"Your honor, this automated world frightens and confuses me"	16:30
adamw	this court rules that you suck it up, princess	16:31
kparal	7. mkrizek worked heavily on posting Bodhi comments. we have now it almost finished, but jlaska proposed some great further enhancements, so the patch will be probably adjusted and posted again this week	16:31
jlaska	adamw: haha	16:31
kparal	mkrizek: do you want to go into detail about his one, or should I continue?	16:31
jlaska	#info mkrizek worked heavily on posting Bodhi comments. It's almost finished, expecting additional changes this week	16:32
jlaska	mkrizek: is probably mad at me ... we just won't let him close that ticket!	16:32
mkrizek	kparal: please do:)	16:32
mkrizek	jlaska: haha	16:32
mkrizek	jlaska: no, those were great comments	16:32
kparal	oh, and he added architecture support into that patch (reading from last week's next steps)	16:33
jlaska	nice, I missed that ... good stuff	16:33
kparal	ok, I think I have covered everything. can't extract any more important information from my last week's log :)	16:34
jlaska	sounds like a busy, but productive week	16:34
jlaska	thanks for the updates	16:34
kparal	you're welcome	16:34
jlaska	any questions/comments for kparal and company before we move on?	16:34
adamw	any news on depcheck?	16:35
jlaska	is wwoods around?	16:35
kparal	wwoods promised some blog post article. apart from that, we didn't touch depcheck in the last week	16:36
kparal	we need the rewritten watcher first	16:36
adamw	ok	16:37
jlaska	iirc, he's also working on http://fedorahosted.org/autoqa/ticket/248	16:37
jlaska	which is related to the blog post I think	16:37
jlaska	basically ... we need a way to keep multiple tests from clobbering each others results	16:38
* Viking-Ice points on if reporters are supposed to be required to comment in bodhi maintainers should be required to provide test cases for component..		16:38
jlaska	Viking-Ice: I don't think we're too far off from having that discussion. imo ... that's related to the ticket/thread adamw kicked off on test@	16:39
jlaska	alright ... anything else on autoqa?	16:39
kparal	nope	16:39
jlaska	okay ... let's dive into open discussion	16:39
jlaska	pjones: still around?	16:39
pjones	yep	16:39
jlaska	if so ... you can go first (so we don't lose you for lunch)	16:39
pjones	okay, well, https://fedoraproject.org/wiki/Features/FourkBSectorBooting	16:40
jlaska	#topic Open discussion - pjones - grub 4kB sectors bugs	16:40
jlaska	#info https://fedoraproject.org/wiki/Features/FourkBSectorBooting	16:40
pjones	this is support for booting from drives with 4kB sectors (as opposed to 512B like normal disks have)	16:40
pjones	it's only on UEFI, but some of the code changes are in common codepaths and they're a bit hairy	16:40
pjones	so I'm just asking people to keep an eye out - if you see new grub installations start and then fail to work, it may be related.	16:41
pjones	I've tested it locally and it seems to work for me, but there's some risk still.	16:41
jlaska	what would fail look like ... grub-install failing, or boot failures?	16:41
pjones	boot failures. most likely you'd see grub start and then who knows what.	16:43
jlaska	okay	16:43
jlaska	would this affect anyone running rawhide today?	16:43
jlaska	s/would/could/	16:43
pjones	the major changes that I'm most concerned with are in the disk buffering.	16:43
pjones	on installs friday and later, yes.	16:43
jlaska	okay	16:43
pjones	we don't re-install grub by default if you do a package upgrade, so if you've got an older install, it probably won't effect you.	16:43
jlaska	any recovery procedure if someone gets bit by this?	16:44
jlaska	aaah	16:44
pjones	boot a rescue image and install an older version of grub, then run grub-install.	16:44
jlaska	okay	16:44
pjones	and, you know, be sure and report it ;)	16:44
jlaska	#info rawhide/f15 now have support for booting from drives with 4kB sectors (as opposed to 512B like normal disks have)	16:44
jlaska	pjones: details details ;)	16:44
pjones	#info only on UEFI	16:44
jlaska	thx	16:45
pjones	the feature hasn't actually been approved yet, but, well...	16:45
fenrus02	are any drives shipping with 4k yet?	16:45
pjones	on the market? no.	16:45
pjones	right now they're NDA-only.	16:45
* fenrus02 nods		16:45
jlaska	pjones: thanks for the heads up	16:46
pjones	which limits our testing a bit. But it works on my box ;)	16:46
jlaska	pjones: when will drives start landing with this support?	16:46
pjones	np.	16:46
pjones	no idea.	16:46
jlaska	heh, surprise!	16:46
pjones	I think that's actually still classified as an "if" rather than a "when", actually.	16:46
jlaska	okay	16:46
jlaska	alrighty ... next topic then	16:47
jlaska	#topic Open discussion - mdomsch - FTBFS	16:47
jlaska	#link http://lists.fedoraproject.org/pipermail/devel/2010-December/146683.html	16:47
jlaska	#link http://lists.fedoraproject.org/pipermail/devel/2010-December/146682.html	16:47
jlaska	mdomsch2: still around?	16:47
jlaska	mdomsch asked, "my FTBFS report includes ~80 bugs that now BFS, I'd appreciate someone _else_ verifying the logs, then closing them"	16:47
mdomsch2	not really	16:47
jlaska	additionally, "begin creating the list of FTBFS bugs that will cause packages to be blocked from F15 at Alpha. starting with those ~110 that are still open from F14, and warning people"	16:48
jlaska	mdomsch2: okay	16:48
jlaska	perhaps nirik might know as well	16:48
jlaska	I'm not extremely familiar with the fallout of FTBFS packages?	16:49
nirik	whats the question? ;)	16:49
jlaska	these are existing packages, that no longer build in rawhide/F15 and will need updates?	16:49
adamw	i'm not sure about the 'will cause packages to be blocked' thing	16:49
adamw	do we block packages for ftbfs?	16:49
dgilmore	adamw: nope	16:49
jlaska	this doesn't strike me as a classic QA task ... but I might be forgetting history here	16:50
nirik	we didn't last cycle... but there was plans to.	16:51
nirik	http://fedoraproject.org/wiki/FTBFS	16:51
mdomsch2	well, we did purge ftbfs pkgs once	16:51
nirik	see last point.	16:51
mdomsch2	we should at alpha compose	16:52
adamw	yeah, this kinda feels more like devel stuff	16:52
nirik	well, or releng...	16:52
mdomsch2	or fesco?	16:52
adamw	or anyone but us!	16:52
adamw	give it to the design team, they'll love it	16:52
mdomsch2	i'd love it if it wzsn't just me driving it	16:53
jlaska	yeah, I don't mean to push off tasks, but I agree ... this feels like a process rel-eng would manage at least?	16:53
jlaska	dgilmore ?	16:53
mdomsch2	it's a meta qa task imho	16:53
adamw	i don't think we ever really expanded QA's remit to 'does it build'	16:54
mdomsch2	is the distro self-hosting?	16:54
dgilmore	jlaska: we dont block packages that ftbfs	16:54
nirik	dgilmore: we have, we just didn't this last cycle. ;)	16:55
mdomsch2	I can take it up elsewhere if qa isn't interested, no problem	16:55
dgilmore	nirik: hrrm, i guess i missed that	16:55
adamw	just feels like it's more likely to result in fixes if it goes under releng/devel	16:55
nirik	mdomsch2: possibly fesco or rel-eng should discuss it?	16:55
mdomsch2	k	16:55
adamw	sure, you can argue the 'find out what builds and what doesn't' bit is QA, but now you have a script that bit is quite trivial: run the script	16:55
jlaska	mdomsch2: it's not so much that we aren't interested ... but more that deciding whether we are self-hosting seems to be a multi-group/fesco thing	16:55
adamw	the hard bit is 'make it build', and that's work for devel	16:55
adamw	to me it feels like the notifications of dependency issues, for instance	16:56
adamw	they go to -devel and aren't part of qa stuff	16:56
jlaska	good comparison	16:56
jlaska	mdomsch2: btw ... is generation of these reports a manual process now?	16:56
abadger1999	Should qa be in charge of the "verify that it now builds before closing bug" portion?	16:57
jlaska	hmmm	16:57
adamw	that feels like needless make-work	16:58
jlaska	that seems unnecesary ... but this might be my lack of understanding around FTBFS	16:58
adamw	it's much more efficient for the developer to just do it when the build succeeds	16:58
adamw	the developer knows what the bug number is, knows when they did the build, and knows what the build number is =)	16:58
jlaska	yeah, the risk of allowing the maintainer to manage this process seems low	16:58
adamw	i'm pretty sure the devs aren't going to start lying about it. and if they do, all that would happen would be a new bug got filed next time the script ran.	16:59
jlaska	good discussion, good ideas	16:59
jlaska	alrighty ... anything else on this topic, or other topics?	17:00
jlaska	#topic Open discussion - <your topic here>	17:00
adamw	i have a quick topic	17:00
jlaska	adamw: take it away	17:00
jlaska	#chair adamw	17:01
zodbot	Current chairs: adamw jlaska	17:01
adamw	critpath test plans!	17:01
jlaska	#topic Open discussion - Critpath test plans	17:01
adamw	jlaska mentioned it earlier, but just to trumpet it	17:01
adamw	https://fedorahosted.org/fedora-qa/ticket/154	17:01
adamw	we need test plans for critical path packages! I'll finish the groundwork today	17:01
adamw	I'm probably going to set up a proposed standard naming scheme for the plans, start filing tickets per component, and write an example test case	17:02
adamw	then it'd be great if others could help contribute test cases	17:02
adamw	the idea being 'this case / set of cases makes sure the critical path-related functionality of this package is working'	17:03
jlaska	adamw: sweet, I think the biggest road block for contributions will be settling in on a standard content convention	17:03
adamw	i don't think the *content* of the test cases needs to be standardized	17:03
adamw	in fact it'd probably be a bad idea	17:03
jlaska	errmm no	17:03
adamw	ah =)	17:03
jlaska	I see what you're saying	17:03
adamw	maybe i misunderstood you	17:03
jlaska	but let's keep things to the 'QA/Test_Case' template for now	17:03
adamw	well, i was thinking of something after that	17:04
adamw	a further prefix	17:04
jlaska	and by convention ... I meant what you refered to earlier about the wiki structure (categories, naming convention etc...)	17:04
adamw	maybe just the package .src.rpm name	17:04
adamw	or packagename_critpath_(whatever)	17:04
jlaska	great phase of the project to play around with different options	17:05
jlaska	the choice is important as it helps testers find content, and allows other tools to link to appropriate content?	17:05
adamw	so we'd have QA/Test_Case_NetworkManager_critpath_connection or something	17:05
adamw	yes, the second is the important bit: bodhi integration is what i'm thinking about	17:05
adamw	as i mentioned it'd be great if bodhi and f-e-k can automatically integrate with the set of test cases for the update in question	17:06
jlaska	spend extra time on the wiki layout/organization step	17:06
adamw	i guess the other thing we can do is have a (sub) category for each component	17:06
jlaska	that'll definitely pay off	17:06
jlaska	adamw: cool, you want this on the agenda for next week then?	17:07
adamw	sure,	17:07
adamw	why not	17:07
jlaska	adamw: I'll bug you by end of week	17:07
jlaska	this will be a nice foot in the door to building our test documentation ... thanks for takin this on	17:08
adamw	npnp	17:08
jlaska	alrighty ... let's close out on 30 seconds	17:08
jlaska	10 seconds ...	17:09
jlaska	alright, thanks all!	17:09
jlaska	#endmeeting	17:09

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!