QA/Meetings/20110207

Attendees

People present (lines said)

1. jlaska (174)
2. adamw (112)
3. kparal (60)
4. Viking-Ice (45)
5. rbergeron (30)
6. maxamillion (13)
7. brunowolff (11)
8. vhumpa (10)
9. tflink (8)
10. wwoods (2)
11. robatino (2)
12. Southern_Gentlem (1)
13. CRCinAU (1)

Unable to attend:

1. Rhe (hopefully sleeping)
2. Hongqing (hopefully sleeping)

Agenda

• Proposed meeting agenda
• MeetBot summary

Previous meeting follow-up

1. jlaska - update Proven_tester#Mentoring_process with information about how to handle FAS group requests without a corresponding ticket
   • See change -- https://fedoraproject.org/w/index.php?title=Proven_tester&diff=219888&oldid=199131

AutoQA update - autoqa-0.4.4 and DevConf + FUDCon updates

Owner

kparal

Summary

jlaska created a great patch for autotest to automatically transfer all autoqa config files from the server to the client

wwoods improved his depcheck test and sent us final version proposal

jskladan posted final version of his new_koji_watcher code

kparal prepared a patch for upgradepath to work with the new post-bodhi-update-batch event (part of the new_koji_watcher stuff)

jskladan created new milestone in AutoQA trac - Finger Food - containing very small tasks suitable for AutoQA newbies -- https://fedorahosted.org/autoqa/milestone/Finger%20Food

kparal created new documentation AutoQA Development -- AutoQA_Development

FUDCon:Tempe_2011 AutoQA slides available at QA:Presentations

Next Steps ...

Complete packaging of compat-Django-1.0.4 security patches and submit for review (jlaska)

Wwoods to review current depcheck patch, and provide feedback

Continue testing and merging of new koji-watcher

Tue, Feb 08 - Alpha 'Test Compose'

Owner

rhe

Summary

See task#19 on Fedora 15 QA calendar (http://poelstra.fedorapeople.org/schedules/f-15/f-15-quality-tasks.html)

Next Steps ...

1. jlaska - ask rel-eng to file tickets for upcoming Alpha milestones
2. jlaska - ping clumens about a new anaconda build for the alpha TC
3. rhe or robatino - Announce ISO availability, and commence testing

Thu, Feb 10 Test Day -- FreeIPA v2

Owner

adamwill

Summary

See https://fedorahosted.org/fedora-qa/ticket/163

No updates in ticket or on wiki yet

Next Steps ...

1. adamw - reach out to dmitri to check-in on FreeIPA test day prep

Fri, Feb 11 - Alpha Blocker Meeting (f15alpha) #3

Owner

jlaska

Summary

The 3rd scheduled blocker bug meeting will be hosted this Friday (see QA:SOP_Blocker_Bug_Meeting).

The f15 Alpha blocker bug is https://bugzilla.redhat.com/show_bug.cgi?id=F15alpha. If you think a bug should block the release of F15 Alpha, set it as blocking that bug, and it will be reviewed at the meeting

The F15 Alpha nice-to-have bug is https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha-accepted. This is for bugs which don't block the release but for which fixes should be allowed through alpha freeze. You can propose bugs for it directly if you're sure they don't meet the blocker requirements; otherwise we usually put things on it when they don't get accepted as blockers

Next Steps ...

1. jlaska - send F15Alpha blocker review announcement
2. All - file bugs and propose as blockers

Testing Fedora 15 in KVM virtual machines

Owner

kparal

Summary

Kamil raised the topic on the mailing list -- see http://lists.fedoraproject.org/pipermail/test/2011-February/096856.html

After discussing the differences in environments, the team agreed that using KVM for testing is acceptable. However, testers and test organizers, should recognize that virt is only one of many environments that may need testing.

Additionally, the desktop validation matrix may need review to account for fallback mode, and other areas to ensure test coverage in multiple environments (bare-metal and virt)

Next Steps ...

1. HELP - Adjustments to the desktop validation matrix (QA:Desktop_validation_testing) may be required to accomodate for fallback mode, and/or other hardware environments. Suggestions/drafts encouraged.

Open discussion - <Your topic here>

GNOME3 Test Day Summary

Owner

Adamwill

Summary

We had the first of three GNOME 3 test days on thursday, it went very well thanks to heroic work by the desktop team to get testing packages ready

We had several dozen testers and lots of juicy bugs reported.

Thanks to everyone who came and tested!!!

rbergeron asked about the process for how bugs are prioritized after the event, how do we know the bugs are prioritized and fixed as needed. Adamwill noted that he would review the bugs filed and recommend moving to the appropriate upstream (or downstream) blocker list.

Next steps ...

Adam expecting to send test summary later today

Best Practices for test day bug reporting

Owner

johannbg

Summary

After discussing the GNOME3 test day practice of filing functionality bugs upstream (bugzilla.gnome.org), and packaging bugs downstream (bugzilla.redhat.com), some debate ensued as to whether multiple bug reporting systems was an impediment to test day contributors.

One idea was to require that all future test days file bugs in the same downstream bugzilla

One idea was to add a caution/note to the test day SOP to noting that multiple bug trackers may be confusing to participants.

The meeting ran over the alloted time, and no agreement was reached

Next steps ...

johannbg agreed to continue discussion on test@lists.fedoraproject.org

HOWTO debug guide requirement

Owner

johannbg

Summary

johannbg suggested we should make it a requirement for test days to have how_to_debug pages ready present for the components that are being tested present and ready before the test

Adamwill noted that adding a suggestion to the test day SOP for a debug guide seems like a good idea

Next steps ...

Propose update to QA/SOP_Test_Day_management

Action items

1. jlaska - ask rel-eng to file tickets for upcoming Alpha milestones
2. jlaska - ping clumens about a new anaconda build for the alpha TC
3. adamw - reach out to dmitri to check-in on FreeIPA test day prep
4. jlaska - send F15Alpha blocker review announcement
5. Viking-Ice - solicit feedback on test@lists.fedoraproject.org to see whether we need to require only bugzilla.redhat.com use during test days

IRC Transcript

jlaska	#startmeeting Fedora QA Meeting	16:00
zodbot	Meeting started Mon Feb 7 16:00:10 2011 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.	16:00
zodbot	Useful Commands: #action #agreed #halp #info #idea #link #topic.	16:00
jlaska	#meetingname fedora-qa	16:00
zodbot	The meeting name has been set to 'fedora-qa'	16:00
jlaska	#topic Gathering	16:00
* kparal waves		16:00
jlaska	Show of electronic hands ...	16:00
* rbergeron raises her robotic arm		16:00
vhumpa	timber	16:00
jlaska	vhumpa: kparal: welcome!	16:01
tflink	* waves	16:01
jlaska	howdy tflink  :)	16:01
jlaska	rbergeron: somehow I'm picturing something from the terminator	16:01
* brunowolff is here		16:01
adamw	yo	16:02
jlaska	brunowolff: adamw: howdy	16:02
* Southern_Gentlem		16:02
jlaska	anyone else ... robatino Southern_Gentlem Viking-Ice ?	16:02
* CRCinAU is physically here...		16:02
robatino	here	16:02
vhumpa	Hey everybody	16:02
rbergeron	jlaska: just pretend i'm one of the girls from the terminator tv series.	16:02
rbergeron	:)	16:02
jlaska	be nice today folks ... this is tflink and vhumpa's first qa meeting	16:03
jlaska	rbergeron: CRCinAU: greetings :)	16:03
jlaska	okay, let's get this party started!	16:03
jlaska	#topic Previous meeting follow-up	16:04
jlaska	I don't believe a meeting was held last week ... at least I didn't see any minutes posted	16:04
jlaska	and there was only one action item from the previous previous meeting	16:04
jlaska	#info jlaska - update Proven_tester#Mentoring_process with information about how to handle FAS group requests without a corresponding ticket	16:04
jlaska	I added a blurb to the end of the mentoring section at Proven_tester#Mentoring_process	16:04
adamw	yay documentation	16:05
jlaska	heh ... +2 sentences ... I'm certainly not a doc guru  :)	16:05
jlaska	that titled goes to our very own adamw	16:05
* maxamillion is here (kinda)		16:05
* jlaska tips hat to maxamillion		16:05
adamw	hey maxa	16:05
jlaska	so comments welcome, otherwise I'm checking that off my list	16:06
jlaska	anything else from previous meetings that we need to review?	16:06
* jlaska sets fuse at 15 seconds		16:06
adamw	don't think so	16:06
jlaska	alrighty ... time for the show	16:06
jlaska	kparal: are you ready to kick things off today?	16:07
kparal	jlaska: sure	16:07
jlaska	#topic AutoQA update - autoqa-0.4.4 and DevConf + FUDCon updates	16:07
jlaska	kparal: take it away my good man	16:07
kparal	ok, so, this is the summary for the last two weeks	16:07
kparal	#info jlaska created a great patch for autotest to automatically transfer all autoqa config files from the server to the client	16:07
kparal	We already had similar hack before, but it didn't work quite right (we used to read config files from current directory). The new one solved a lot of issues for us and in the future we could use this approach for transferring also the AutoQA library (and therefore won't have to maintain the clients at all) - I already created a proposal for that, we will want to deal with it in the next+1 release.	16:08
jlaska	kudos to lmr for that patch idea	16:08
kparal	#info wwoods improved his depcheck test and sent us final version proposal	16:08
kparal	It is not included yet, I have posted some feedback on that (there were a few tracebacks) and we haven't received a reply yet. But generally it looks in a quite good shape.	16:08
kparal	#info jskladan posted final version of his new_koji_watcher code	16:09
kparal	The new Koji watcher code should obsolete the old koji watcher and also the old Bodhi watcher. I would almost agree to include it in the master, but the output of the new Koji watcher and of the old Bodhi watcher still differs a little. We believe it's caused by a bug in Bodhi that should be fixed by now, but not deployed yet. Confirmation from lmacken is needed.	16:09
jlaska	is wwoods lurking? not sure if that's on his radar	16:09
jlaska	kparal: how much does it differ?	16:10
kparal	lmacken seems to be not available right now, pitty	16:10
jlaska	do we gain or lose tests using the new watcher?	16:10
kparal	jlaska: well, both :)	16:10
* wwoods is lurking		16:10
jlaska	kparal: heh ... isn't that always the case! :)	16:10
wwoods	I'll read the followup comments and see what I can do	16:10
kparal	we gained some more, which is a good thing. the old bodhi watcher was flawed	16:10
kparal	but we also lost some, which is I believe caused by Bodhi bugs	16:10
kparal	I need confirmation from lmacken that the fix has not been pushed to production yet	16:11
jlaska	wwoods: sweet, thank you ... I've got your branch setup and can walk through some sample test runs if needed	16:11
kparal	if it was, then we need to look at it more	16:11
jlaska	kparal: okay	16:11
kparal	and after the fix is pushed, we need to do the testing again	16:11
kparal	so, lmacken needed for us to be certain :)	16:12
jlaska	okay ... we'll send out a search party after meeting	16:12
kparal	alright, let's move on next	16:13
kparal	#info kparal prepared a patch for upgradepath to work with the new post-bodhi-update-batch event (part of the new_koji_watcher stuff)	16:13
kparal	I have the code prepared, but it was not yet posted for review, because we need to accept new_koji_watcher first. So, more on this later.	16:13
jlaska	points for preparedness!	16:13
kparal	:)	16:13
kparal	#info jskladan created new milestone in AutoQA trac - Finger Food - containing very small tasks suitable for AutoQA newbies	16:13
kparal	That will surely come handy soon.	16:13
kparal	#link https://fedorahosted.org/autoqa/milestone/Finger%20Food	16:13
kparal	and the last one:	16:14
kparal	#info kparal created new documentation AutoQA Development	16:14
jlaska	I like that idea, I added a few items to that roadmap last week	16:14
kparal	It describes an easy way how to setup full AutoQA development environment (your machine, AutoQA server, AutoQA client). Ideal for new guys I believe (and also for me to remember the setup).	16:14
kparal	#link AutoQA_Development	16:14
kparal	that's all the news I believe	16:15
tflink	as someone who is following that doc, would you prefer suggestions if/when I run into issues or should I just edit the wiki pages?	16:15
jlaska	ooh, very handy	16:15
kparal	tflink: as you seem fit, both approaches are just fine	16:15
vhumpa	Looks very nice to start up, I'll check it out	16:15
tflink	kparal: OK, thanks	16:15
adamw	tflink: i'd say in general if you see something small that's just inaccurate / out of date, fix it	16:16
kparal	tflink: you can discuss it on Talk page, at autoqa-devel, over IRC, or just edit it if you find an error :)	16:16
adamw	for bigger stuff or if you're not quite sure, find an adult first :P	16:16
* jlaska looks around for one		16:16
adamw	as in, someone who knows about what you're editing	16:16
adamw	sometimes that may even be jlaska (terrifying as the prospect sounds(	16:17
jlaska	god help us	16:17
kparal	tflink: certainly don't be afraid to harass me with any autoqa questions you have, anytime	16:17
tflink	adamw: you mean that making big changes without clearing it with someone would be a bad idea? :-P	16:17
adamw	tflink: sometimes!	16:17
adamw	:P	16:17
adamw	main thing, if you're pretty sure about your change, just go ahead and do it, anything else creates unnecessary bureaucracy, the ROOT OF ALL EVIL	16:18
jlaska	kparal: Using that handy openoffice.org template you used for your AutoQA presentation, I gave a brief update on the status and roadmap of autoqa at FUDCon Tempe. The slides are fairly brief, I spent more time talking and discussing with participants	16:18
tflink	kparal: will do, I've been looking through the code and am just getting to the point where I can start asking intelligent questions	16:19
jlaska	#info FUDCon:Tempe_2011 AutoQA slides available at QA:Presentations	16:19
vhumpa	kparal: i wish to get to that point by this week :-)	16:19
kparal	jlaska: were there some interesting questions/responses from the audience?	16:19
jlaska	I had a lot of great conversations around AutoQA and autotest as well. I'll provide more in a blog/trip_report later this week	16:20
kparal	ok, nice	16:20
jlaska	kparal: requests for new test wrappers for existing tests (tickets filed), dmalcolm requested his rpmlint whitelisting idea again and showed some sample code, some discussion around ensuring deps don't 'splode for some SIGs (namely the server SIG)	16:21
jlaska	kparal: anything you wanted to highlight for the developer conference this weekend ... or should we save those details for later?	16:22
kparal	well	16:22
kparal	there will be a Red Hat developer conference this weekend in Brno: DeveloperConference2011	16:22
kparal	I will be giving a short AutoQA workshop there	16:22
kparal	I intend to show people how to create a really simple AutoQA test	16:23
kparal	if I have any interesting and usable outputs of that workshop, I'll surely publish it	16:23
jlaska	looks like lennart will be there ... any chance he has thoughts on some sort of systemd-lint tool (or something analogous to our initscripts tests)?	16:24
kparal	jlaska: I can surely ask him	16:24
* Viking-Ice joins in late had a meeting conflict		16:24
jlaska	kparal: thanks, if you have time to grab him for some input	16:24
kparal	(but I should study systemd first, so I don't look too stupid when asking about it :) )	16:24
jlaska	Viking-Ice: welcome	16:25
jlaska	kparal: I'd basically want to know if there is anything we can do to lint a package that includes a systemd script. Or how we should extend the initscripts tests to support systemd	16:25
jlaska	(nothing concrete, just exploring ideas)	16:25
kparal	jlaska: sure, great question	16:26
jlaska	quite a lot of updates for the last 2 weeks ... anything else to note?	16:26
kparal	that's all from me	16:26
jlaska	kparal: sweet, thanks Kamil	16:26
jlaska	a few reminders ...	16:27
jlaska	#topic Tue, Feb 08 - Alpha 'Test Compose'	16:27
jlaska	#info task#18 on Fedora 15 QA calendar (http://poelstra.fedorapeople.org/schedules/f-15/f-15-quality-tasks.html)	16:27
jlaska	#undo	16:27
zodbot	Removing item from minutes: <MeetBot.items.Info object at 0x2b02592df9d0>	16:27
jlaska	#info task#19 on Fedora 15 QA calendar (http://poelstra.fedorapeople.org/schedules/f-15/f-15-quality-tasks.html)	16:27
jlaska	there we go	16:27
Viking-Ice	I think the biggest question here is anaconda status so far any livecd/usb installation is a no go	16:27
Viking-Ice	will that work on alpha?	16:28
Viking-Ice	installing that is	16:28
jlaska	It should, but that's why we have these milestones to identify and prioritize the pain points	16:28
jlaska	I'm not sure if rel-eng has filed a ticket yet for the Alpha deliverables ... I'll check into that after the meeting	16:28
* adamw notes we seem to have missed two alpha blocker meetings already		16:28
adamw	whoops	16:28
brunowolff	I have been looking at bug 672265.	16:29
adamw	or, I mean...you guys didn't show up!	16:29
brunowolff	It doesn't seem to be an alpha blocker.	16:29
Viking-Ice	I did mentally not physically	16:29
Viking-Ice	showing up that is	16:29
maxamillion	adamw: for shame!	16:29
adamw	if that's the liveinst fails bug and it's not marked as an alpha blocker, mark it	16:29
jlaska	adamw: yeah ... that makes two of us :(	16:29
jlaska	I don't see a ticket for this yet at https://fedorahosted.org/rel-eng/report/3	16:29
brunowolff	I'd like to make it NTH, but I am not sure what is the right way to do that.	16:29
* rbergeron wonders if alpha blocker meetings are things she's supposed to be doing and completely failed		16:29
adamw	rbergeron: we can all share the fail!	16:30
rbergeron	adamw: GROUP HUG	16:30
jlaska	rbergeron: we can chat after ... poelcat would drive a lot of these, and we can certaily use help balancing the load :)	16:30
rbergeron	jlaska: sounds good	16:30
rbergeron	i will be here :)	16:30
rbergeron	(and elsewhere)	16:30
adamw	sometimes poelcat drove, sometimes I did; I don't think it's officially part of your Job Description though, just something he did to spread the load	16:30
jlaska	#action jlaska - ask rel-eng to file tickets for upcoming Alpha milestones	16:30
brunowolff	If it really is a blocker, then the criteria should be changed. The install criteria only applies to install only images,	16:30
brunowolff	according to the wiki.	16:31
brunowolff	I'd like to get a dracut or dm expert to look at that bug.	16:31
adamw	brunowolff: no, it doesn't	16:31
brunowolff	Anyway I'll add it to the alpha blocker tracking bug.	16:31
adamw	"The installer must boot (if appropriate) and run on all primary architectures from default live image, DVD, and boot.iso install media "	16:31
adamw	is one of the Alpha criteria	16:31
maxamillion	solid criteria	16:32
jlaska	robatino are you available to assist rhe with wiki and announcement magic for the validation events?	16:32
adamw	and then "The installer must be able to complete package installation with the default package set for each supported installation method "	16:32
brunowolff	OK, I was looking through the detail breakout and didn't set it there. And in the breakout it specifically mentions	16:32
adamw	yeah, basically, the alpha criteria imply that any complete fail bug in the installer is a blocker.	16:32
brunowolff	install only images. That's a bit confusing.	16:32
jlaska	I suspect we'll need a new anaconda build shortly ... I'll ping clumens	16:33
robatino	jlaska: yes - i haven't been paying that much attention lately - is the procedure the same as before?	16:33
jlaska	#action jlaska - ping clumens about a new anaconda build for the alpha TC	16:33
jlaska	robatino: yeah should be	16:33
Viking-Ice	jlaska: yeah his last one was from 25 jan I believe	16:33
adamw	brunowolff: the only one that mentions them specifically is #3, and that's because live images don't boot to anaconda	16:33
jlaska	okay ... so $topic will offer some excitement by way of bugs this week	16:33
jlaska	any other questions on this topic?	16:33
adamw	er, i mean, don't offer install options	16:33
jlaska	robatino: if I haven't said it enough ... thanks for helping with the wiki/announce for these events :)	16:34
jlaska	okay ... next up ...	16:34
adamw	it does look like something's missing there, though, i'm sure there's supposed to be a criterion about the live image's boot menu	16:34
adamw	will investigate later	16:34
jlaska	#topic Thu, Feb 10 Test Day -- FreeIPA v2	16:34
* jlaska looks in TRAC to see who requested this event		16:34
brunowolff	The bug is now a blocker for F15Alpha.	16:34
jlaska	#link https://fedorahosted.org/fedora-qa/ticket/163	16:35
jlaska	looks like this came from Dmitri Pal	16:35
jlaska	anyone have time to reach out to dpal to see how test day prep is coming along?	16:35
jlaska	if not ... I'm happy to ping	16:35
rbergeron	btw: this is a feature that is going to FESCo on wednesday for feature approval - they forgot to move it to featurereadyforwrangler on the wiki.	16:36
Viking-Ice	this one requires a bunch of docs + debug info	16:36
Viking-Ice	as in we need to provide setup instruction for each service for the test day I believe	16:36
Viking-Ice	and debug info would be good to have/finish writing that up	16:37
jlaska	Viking-Ice: likely, I'd like to see what they wanted to accomplish for the test day	16:37
adamw	i can get in touch with dmitri	16:37
Viking-Ice	adamw: feel free to cc me to that	16:37
jlaska	adamw: thanks	16:37
adamw	Viking-Ice: roger	16:37
adamw	can you #action it for me jlaska?	16:37
jlaska	#action adamw - reach out to dmitri to check-in on FreeIPA test day prep	16:37
adamw	yay	16:37
maxamillion	<3 meetbot	16:38
jlaska	we can always post-pone the event if sufficient time isn't available to properly setup this event	16:39
adamw	yup	16:39
Viking-Ice	agreed	16:39
jlaska	anything else here ...	16:39
jlaska	#topic Fri, Feb 11 - Alpha Blocker Meeting (f15alpha) #3	16:40
jlaska	we kind of discussed this already, but just wanted to remind everyone about the Alpha blocker review scheduled for this Friday	16:40
adamw	the first two went so well!	16:40
jlaska	they certainly did!	16:40
jlaska	EFAIL	16:40
jlaska	adamw: rbergeron: Unless any objections, I'll grab announcing and meetbot duties for the event this Friday?	16:41
rbergeron	works for me. I will watch and learn ;)	16:41
rbergeron	or pitch in if someone throws something at me.	16:41
adamw	sure	16:41
jlaska	learn how *not* to host them :)	16:41
jlaska	#link QA:SOP_Blocker_Bug_Meeting	16:41
jlaska	not sure there is anything else to add here	16:42
jlaska	is there a BugZappers meeting this week?	16:42
jlaska	do we need to remind folks about F15Alpha and the nice-to-have list?	16:42
adamw	can't hurt	16:43
adamw	the f15 Alpha blocker bug is https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha	16:43
jlaska	#action jlaska - send F15Alpha blocker review announcement	16:43
adamw	if you think a bug should block the release of F15 Alpha, set it as blocking that bug, and it will be reviewed at the meeting	16:43
adamw	the F15 Alpha nice-to-have bug is https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha-accepted	16:43
jlaska	#link https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha	16:43
adamw	that's for bugs which don't block the release but for which fixes should be allowed through alpha freeze	16:44
adamw	you can propose bugs for it directly if you're sure they don't meet the blocker requirements; otherwise we usually put things on it when they don't get accepted as blockers	16:44
jlaska	... and fixing them isn't considered invasive to the release	16:45
jlaska	s/invasive/disruptive/	16:45
jlaska	adamw: thanks for the links	16:45
jlaska	next up, a topic kparal raised on the list ...	16:45
jlaska	#topic Testing Fedora 15 in KVM virtual machines	16:45
adamw	good one	16:45
kparal	yes, there was a short discussion on the list	16:46
jlaska	I believe Kamil wanted to note that using KVM guests no longer exercises the intended default desktop user experience	16:46
kparal	yes, and I want to ask what whether our approach to KVM testing changed and how	16:46
jlaska	#link http://lists.fedoraproject.org/pipermail/test/2011-February/096856.html	16:46
adamw	we may have to tweak a few notes on criteria pages &c	16:46
jlaska	perhaps, I suspect those will fall out during the blocker reviews?	16:47
adamw	it's probably worth talking to virt team about acceleration passthrough plans	16:47
Viking-Ice	this only applies to Gnome desktop btw	16:47
Viking-Ice	testing	16:47
jlaska	Viking-Ice: yes, thanks	16:47
adamw	i believe something's queued up for spice	16:47
jlaska	kparal: my take is I still plan to test with virtualization, but one needs to understand what is being tested since the virt environment may not meet the expected results for a given case	16:48
jlaska	so testing with virt alone for the entire release isn't sufficient	16:48
kparal	well, it's clear that we can't test default desktop in KVM. we can test just the fallback mode, and not to the full extent	16:48
adamw	it'll be fine for exercising the live installer	16:48
Viking-Ice	jlaska: *cough* it never is	16:48
adamw	it'll be useless for the desktop validation testing	16:48
jlaska	Viking-Ice: *exactly*	16:49
kparal	I believe we can still do installation testing, but we must be clear what "system boot successfuly" means	16:49
maxamillion	adamw: well, that's true	16:49
adamw	as things stand all gnome desktop validation will have to be done on raw metal	16:49
vhumpa	kparal: Looks like at that point we'll simply have to use bare hardware	16:49
jlaska	how does the fallback mode different on bare-metal vs virt?	16:49
maxamillion	adamw: isn't bare metal the prefered case for all desktop validation? just so we can test with different hardware drivers and such?	16:49
adamw	kparal: that criterion is really intended for the _installer_ per se - does the installer properly render the installed system bootable	16:50
adamw	kparal: bugs in the actual desktop installed are meant to be covered by other criteria	16:50
adamw	maxamillion: sure, doing it in virt can be handy for some testing though	16:50
adamw	desktop menus koff koff	16:50
kparal	jlaska: quoting JBG: For any Gnome related test days you cant use KVM since one of the	16:50
kparal	information we need to gather from test days for developers is that if	16:50
kparal	users are getting either a shell or fallback mode or not and those	16:50
kparal	reporters that aren't getting shell or fallback mode will need to file a	16:50
kparal	bug providing their smolt profile so developers can look deeper into	16:50
kparal	what hw is failing to reach shell or fallback mode.	16:50
jlaska	maxamillion: adamw: virt is another environment, just like bare-metal	16:50
maxamillion	jlaska: I imagine it will be the same if it works, but likely if everyone is testing on virt and there's some issue with the hand off to fall back mode based on some specific graphics card (or $other factor) it would be missed with virt	16:51
jlaska	maxamillion: definitely	16:51
Viking-Ice	yup any Gnome testing needs to be performed on bare metal	16:51
maxamillion	adamw: right, I do all my pre-alpha rawhide testing in virt because I don't have enough spare hardware to have rawhide eating my hard drives contents	16:52
maxamillion	jlaska: +1	16:52
jlaska	kparal: hmm ... I'll need to confirm with jrb, since I think virt is still valid for testing fallback support ... but *not* for testing the default GNOME experience	16:52
adamw	it	16:52
adamw	it's valid for testing the fallback experience	16:52
kparal	jlaska: I believe that too	16:52
adamw	as noted, we do need bare metal testing to make sure the fallback _mechanisms_ work	16:52
Viking-Ice	yup which is pretty much what I said in that thread	16:52
jlaska	indeed	16:52
adamw	kvm is just one fallback case out of...zillions	16:52
jlaska	well said	16:53
kparal	ok, we all understand then :)	16:53
Viking-Ice	:)	16:53
vhumpa	no problem for me to test on raw hw, as soon as the rawhide is properly installable	16:53
kparal	vhumpa: good point :)	16:53
adamw	heh	16:53
jlaska	proposed - #agreed Using KVM for testing is still supported and needed. However, recognize that the virtual environment is just one of *many* environments that needs testing, and doesn't offer testing ofthe default GNOME3 user experience	16:53
maxamillion	Xfce Spin works in full featured mode on KVM! </shameless plug>	16:54
jlaska	+1, -1	16:54
kparal	should we alter the desktop validation matrices?	16:54
maxamillion	>.>	16:54
kparal	gnome-shell vs fallback mode?	16:54
Viking-Ice	vhumpa yeah I'm experiencing bugs on my update-to-rawhide that I did not see on the compose ( live usb ) that our newly crowned insomnia king compose for Gnome test day:)	16:54
maxamillion	kparal: might be a good idea	16:54
adamw	kparal: yeah, proposals welcome	16:55
jlaska	presently, we leave it open as an exercise for the user	16:55
adamw	note that i wrote a metric assload of new test cases for the gnome test day, any of which we can slot in as validation tests if it seems appropriate	16:55
jlaska	do we want to exhaustively list all envirements, or just bare-metal and virt, or list several fall-back tests	16:55
adamw	we could just have an extra column - one for gnome shell, one for fallback	16:55
vhumpa	viking-ice: yes, we can't have somebody loose all sleep over life cd's :)	16:56
jlaska	adamw: yes!	16:56
adamw	but yeah, let's not hash it out here	16:56
jlaska	agreed	16:56
adamw	on-list for proposals?	16:56
jlaska	#agreed Using KVM for testing is still supported and needed. However, recognize that the virtual environment is just one of *many* environments that needs testing, and doesn't offer testing of the default GNOME3 user experience	16:56
jlaska	#info Adjustments to the desktop validation matrix (QA:Desktop_validation_testing) may be required to accomodate for fallback mode, and/or other hardware environments	16:57
jlaska	at a mimimum, I think'll need a GNOME specific fallback test	16:57
jlaska	okay ... we are approaching the hour mark	16:57
jlaska	time for open discussion ...	16:57
jlaska	#topic Open discussion - <Your topic here>	16:57
adamw	so, we had a lil' test day last week you may have heard of	16:58
maxamillion	adamw: +1 to on-list for proposals	16:58
jlaska	adamw: did you want to discuss that here?	16:59
adamw	i was gonna do a wrap-up	16:59
Viking-Ice	I got one.. I think we should make it a requirement for test days to have how_to_debug pages ready present for the components that are being tested present and ready before the test day is held :)	16:59
jlaska	#topic GNOME3 Test Day wrap-up	16:59
adamw	#link Test_Day:2011-02-03_GNOME3_Alpha	16:59
jlaska	Viking-Ice: that seems like a nice-to-have to me ... it's not entirely easy to create those pages, in addition to the other tests and wiki pages required for the test day	17:00
adamw	we had the first of three GNOME 3 test days on thursday, it went very well thanks to heroic work by the desktop team to get testing packages ready	17:00
adamw	*waves big #topic stick*	17:00
* jlaska remains quiet until #topic has concluded		17:00
adamw	we had several dozen testers and lots of juicy bugs reported	17:00
adamw	thanks to everyone who came and tested	17:00
jlaska	+1	17:01
adamw	i'll do a proper wrap-up mail soon	17:01
* rbergeron has a question?		17:01
adamw	question away!	17:01
jlaska	sweet, thanks Adam. I wasn't able to participate during the event ... but the wiki is quite active	17:01
rbergeron	possibly noob question. even very likely.	17:01
adamw	set phasers to laugh	17:02
rbergeron	So all of the bugs found were getting reported against GNOME's bz, is that correct?	17:02
adamw	most of them	17:02
Viking-Ice	really I filed mine against rh bugzilla	17:02
adamw	things that are clearly fedora packaging issues went to RH bugzilla	17:02
vhumpa	adamw: thanks for the day, I think that was particularly nice test-day for a newbie	17:02
adamw	Viking-Ice: the notes said to use GNOME bugzilla, but never mind :) it's not hugely important	17:02
adamw	vhumpa: glad it was good for you@	17:02
rbergeron	I guess I'm confused as to how we track those back to Fedora to see if things are fixed. Is it just "we hope testing goes better next time" or do we actually keep track of the bugs we filed over there or ... ?	17:03
rbergeron	or not that testing goes better, since it went pretty well,	17:03
jlaska	were any of the upstream bugs added to the shell blocker list?	17:03
rbergeron	but we hope that next round of testing yields less bugs.	17:03
Viking-Ice	I personally am against letting reporters run around the half the internet filing in each and every bugzilla track instance out there	17:03
adamw	haven't done all the tracking yet	17:03
adamw	Viking-Ice: this was what the developers requested, when we asked	17:03
* rbergeron is'nt harassing, just curious about processes i'm unfamiliar with thus far :)		17:03
adamw	we checked with j5, caillon, and mclasen	17:03
adamw	rbergeron: we have a very dumb little script i hacked up which runs over the test day page and pulls out any bug URLs	17:04
adamw	rbergeron: i'll have to tweak it slightly to look for gnome as well as RH bz for this day	17:04
jlaska	Viking-Ice: I don't think we asked them to file where ever they wanted ... it was requested that upstream issues be reported upstream, and packaging be reported against Fedora.	17:04
vhumpa	yes, i think i also filled a bug with RH, that should have been to Gnome	17:04
adamw	rbergeron: you can fire off that script whenever you like and track the changes to the reported bugs	17:04
Viking-Ice	I think we need to get a larger reporters feel about running having upstream accounts for components we test against	17:05
rbergeron	adamw: that sounds FANCY. :)	17:05
adamw	rbergeron: for X test days, I've usually reviewed how bugs reported were handled later in the cycle, check the archives for some of those mails; we can do the same for GNOME and probably will	17:05
Viking-Ice	s/running/	17:05
Viking-Ice	if maintainers request that we should reject it is my take on it	17:05
adamw	it didn't seem to cause any problems.	17:05
jlaska	Viking-Ice: hrmm, probably something we can hash outside of meeting ... but I don't see a reason to mandate that upstream bugs should be filed downstream	17:06
adamw	rbergeron: the 'script' is at the bottom of QA/SOP_Test_Day_management , it's really not fancy at all	17:06
rbergeron	adamw: i shall check it out :)	17:06
tflink	adamw: do you know how many bugs were filed in RH bugzilla that should have been filed with GNOME?	17:06
adamw	it just pipes the test day page through a hideous heath robinson arrangement of text processing tools :P	17:06
rbergeron	anyway. SORRY FOR NOOB DISTRACTION :)	17:06
adamw	tflink: nope. again, i haven't been through all the post-event teardown yet	17:06
adamw	i needed to do dumb things like sleep and learn to snowboard	17:06
jlaska	details	17:06
jlaska	alright ... let's start wrapping up	17:07
adamw	i'll probably do it today.	17:07
jlaska	anything else on GNOME3 test day?	17:07
Viking-Ice	jlaska: we are downstream and requiring that reporters have upstream bugzilla account and equivalent does not scale. one account in our bugzilla should suffice	17:07
jlaska	adamw: roger, thanks	17:07
adamw	Viking-Ice: it's not something i'd expect to happen a lot.	17:08
jlaska	Viking-Ice: mandates like that don't scale imo ... test days are designed to be flexible to meet the needs of maintainers and packagers	17:08
adamw	the gnome events are somewhat special as they're really combined gnome3 / fedora 15 test days, and were billed as such.	17:08
jlaska	if having an upstream bugzilla account was a big blocker, we could easily just file tickets to bugzilla.redhat.com and later migrate them (by hand or script)	17:08
jlaska	but it doesn't sound like it was a tremendous blocker to participation	17:08
jlaska	#topic Open Discussion - <your topic here>	17:09
adamw	yeah, we didn't get any questions/complaints about it during the day.	17:09
Viking-Ice	jlaska: now comes KDE test day and reporters need to have KDE upstream account etc..	17:09
Viking-Ice	pandora box opening	17:09
Viking-Ice	you allow one we have to allow them all	17:09
jlaska	perhaps, we'll deal with that when we get there	17:09
vhumpa	Loking forward to that :)	17:09
Viking-Ice	or take preventing measures	17:09
jlaska	we allow what maintainers ask for	17:09
Viking-Ice	why are we then using our bugzilla et al	17:10
tflink	adamw: I can't speak for anyone else, but I completely missed the upstream account requirement. That may have affected feedback	17:10
jlaska	Viking-Ice: off-topic ... I don't think we really need to answer that, but I'll be happy to on #fedora-qa	17:10
adamw	tflink: yeah, some bugs may be in RH bugzilla that probably shouldn't be. again, it's not a huge deal breaker or anything.	17:10
adamw	(practically speaking, the owners of the bugs will be the same people half the time anyway.)	17:11
Viking-Ice	jlaska: why off topic	17:11
Viking-Ice	?	17:11
jlaska	Viking-Ice: discussing why we have a bugzilla.redhat.com is not relevant to this discussion	17:11
Viking-Ice	ok heres a topic for the feeding should we or should we not allow maintainers to require direct upstream filing and accounts for test days	17:12
Viking-Ice	you can set that as info gather feed back from test list	17:12
jlaska	#info should we or should we not allow maintainers to require direct upstream filing and accounts for test days	17:12
jlaska	to me, that feels like a distraction to debate over	17:13
Viking-Ice	?	17:13
adamw	yeah	17:13
Viking-Ice	distraction of what	17:13
Viking-Ice	exactly	17:13
adamw	we can't enforce a requirement like that, anyway	17:13
Viking-Ice	yes we can we can drop such request from maintainers	17:14
adamw	it just doesn't feel like something particularly productive to worry about, to me	17:14
Viking-Ice	and require them to actually use our bugzilla instance	17:14
jlaska	Viking-Ice: it's not clear what we lose or gain by investing in that requirement	17:14
adamw	we can add a note to the SOP explaining that an external bugzilla request is a barrier to entry and to avoid it if possible	17:14
jlaska	it seems like "process" for the sake of process	17:14
jlaska	was it a barrier?	17:14
adamw	but phrasing things as requirements and Thou Shalt Nots just doesn't feel terribly helpful to me	17:14
jlaska	bugs got filed	17:14
adamw	anyone else have an opinion?	17:14
adamw	jlaska: in theory it is, and it's hard to prove that it's not	17:14
Viking-Ice	I want this topic to reach the wider audience please	17:14
jlaska	the preference was to file upstream, worst case ... they were filed in bugzilla.redhat.com	17:15
adamw	jlaska: we do get reports where people just don't file bugs, and as tflink said, some people seem to have filed in RH bugzilla not upstream	17:15
jlaska	Viking-Ice: okay, do you want to take this to list@ please?	17:15
adamw	so it's hard to say definitively	17:15
Viking-Ice	jlaska: sure throw that task at me	17:15
jlaska	I'm open to being wrong, it happens a lot :)	17:15
* kparal votes to solve this on test list		17:15
adamw	check the last (currently) entry on the page, for e.g. - lots of notes about interesting-looking bugs, no reports filed	17:15
* kparal can't respond, you're typing too fast :)		17:15
jlaska	this just feels like another bike shed discussion	17:15
* rbergeron hands jlaska the blue paint		17:15
jlaska	I'd definitely want feedback from those who have organized test days, or maintainers who have pitched test days	17:16
adamw	yeah, i feel bike shed-y.	17:16
jlaska	#action Viking-Ice - solicit feedback on test@lists.fedoraproject.org to see whether we need to require only bugzilla.redhat.com use during test days	17:16
Viking-Ice	jlaska: this does not scale requiring or requesting that reporters file upstream bugs reacquires them to have in most case upstream account	17:17
adamw	i think viking's other suggestion was a lot more interesting, though again, shouldn't be phrased as a requirement.	17:17
jlaska	suggestion, yeah	17:17
* jlaska missing the scaling problem		17:17
adamw	me too	17:17
jlaska	what doesn't scale to me is having only a few people pitch and host test days	17:17
jlaska	not which bug tracker to use	17:17
adamw	it's not like we have people showing up to every single test day and bugzilla accounts take up 1GB of hard disk space each	17:17
adamw	i have hundreds of the things	17:17
adamw	took me two minutes each to sign up for	17:18
Viking-Ice	I think I have about 25 upstream accounts already only for the purpose to file upstream against various components <sigh>	17:18
jlaska	when developers/maintainers complain that they are having a hard time tracking bugs from the test days ... then we get dig deeper	17:18
jlaska	alright ... the fuse has been set for 1 minute	17:18
adamw	it's a very small added requirement for any one particular test day's testers, that's about all	17:18
rbergeron	if you're interested in signing up for an hour or two of help, the extra 2 minutes to get an upstream bz account shouldn't hurt. what hurts is when upstream doesn't know all of the possible places to check, where duplicate bugs are reported, and things just wind up not getting fixed, would be my guess.	17:18
jlaska	yes, well phrased	17:19
tflink	is a noob, but votes for whatever is easiest for testers and people running the test days - more participation == better	17:19
Viking-Ice	it's the maintainers/packager job to keep tab on that stuff	17:19
jlaska	30 seconds ...	17:19
rbergeron	i think we have a vested interest in helping to make sure that GNOME goes as smoothly as possible out the door.	17:19
adamw	okay, now you're getting into the wider philosophical argument about whether users should file bugs downstream and maintainers move them upstream, which I REALLY don't want to get into here	17:19
jlaska	bingo	17:19
rbergeron	if that means we ask people to kindly file things upstream, then so be it. :)	17:19
adamw	it goes around all the time on the lists and never goes anywhere and just wastes everyone's time	17:20
adamw	so i'm not gonna engage with that discussion...	17:20
jlaska	new topic ... or end of meeting gang	17:20
rbergeron	particularly if it makes their lives #	17:20
rbergeron	# Features/LZMA for Live Images	17:20
rbergeron	oh, that's what i'm working on.	17:20
rbergeron	... easier. :)	17:20
jlaska	this is the longest 2 minutes ever	17:20
jlaska	10 seconds until #endmeeting	17:20
* rbergeron hands jlaska a timer		17:20
jlaska	Alright, thanks everyone for input+discussion+debate	17:21
jlaska	let's continue on the list as needed	17:21
jlaska	I'll follow-up to the list with minutes, later today	17:21
jlaska	#endmeeting	17:21

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!