QA/Meetings/20100315

Attendees

People present (lines said)

• jlaska (109)
• adamw (45)
• kparal (22)
• daumas (5)
• zodbot (4)
• Viking-Ice (4)

Regrets:

• User:liam
• User:rhe
• User:wwoods
• User:jskladan

Agenda

• Proposed meeting agenda
• MeetBot summary

Previous meeting follow-up

1. maxamillion seeking input from the Mentors group for guidance on mentor responsibilities
2. continued testing of bug#567346 to further isolate the failure conditions
   • This issue is being tracked as a F13Beta blocker, see blocker meeting recap
3. jlaska - update AutoQA_PatchProcess to include fedorapeople.org git repo
   • AutoQA_PatchProcess#Private_Branch

Fedora 13 test status

Upcoming test milestones:

• Pre-beta acceptance test - awaiting new anaconda build before sending through AutoQA - https://fedorahosted.org/rel-eng/ticket/3495
• 2010-03-18 - Beta test compose
• 2010-03-18 - Palimpsest & udisk improvements Test Day
• 2010-03-19 - F13Beta blocker review #2

proventesters

Jlaska asked the group what tasks remain in order to move forward on defining the proventesters group and workflow.

• Adam noted that Jkeating may be working several tasks on this front.
• Jlaska directed those interested in defining providetesters, to provide feedback to maxamillion's email.

Privilege Escalation test

Adamw reminded the group that we've not yet defined test cases to validate the recently approved Privilege_escalation_policy.

• Adamw felt the first step is to come up with the script to identify privesc-relevant packages.
• Kparal asked whether these would be automated tests, or tests more like the desktop validation plan. Discussion followed on the appropriate place to automate these
• The group discussed candidate packages to inspect for polkit changes, including polkit configs, setuid binaries, certain d-bus config files, consolehelper files and pam configs.
• Adam took action to discuss a privileged package detection script with wwoods

Open discussion - <Your topic here>

Update Policy Proposals

Kparal clarified that current package update proposals require that deps/conflicts tests occur whenever a package enters updates and updates-testing. Kparal also discussed concerns around AutoQA being a bottleneck for packages.

AutoQA packaging

Adamw asked how AutoQA packaging was progressing.

• Jlaska noted that packaging wasn't moving, and that a FAD is being planned to help work down the list of java %buildrequires. More information will be sent to [1] later in the week.

Upcoming QA events

• 2010-01-21 - Pre-Alpha Rawhide Acceptance Test Plan #1
• 2010-01-28 - Pre-Alpha Rawhide Acceptance Test Plan #2
• 2010-02-04 - Pre-Alpha Rawhide Acceptance Test Plan #3
• 2010-02-04 - NFSv4 Test Day
• 2010-02-05 - Alpha Blocker Meeting (F13Alpha) #1 (recap)
• 2010-02-11 - Test Alpha 'Test Compose' (boot media testing)
• 2010-02-12 - Alpha Blocker Meeting (F13Alpha) #2 (recap)
• 2010-02-12 - Alpha Test Candidate verification (announcement)
• 2010-02-18 - Alpha Release Candidate verification
• 2010-02-18 - Color Management test day
• 2010-02-19 - Alpha Blocker Meeting (F13Alpha) #3 (recap)
• 2010-02-24 - Alpha Go/No-Go Meeting (20:00 EST)
• 2010-02-25 - Yum Langpack plugin test day
• 2010-03-10 - Pre-Beta Acceptance Test Plan #1
• 2010-03-12 - Beta Blocker Meeting (F13Beta) #1 (recap)
• WE ARE HERE
• 2010-03-18 - Test Beta 'Test Compose'
• 2010-03-19 - Beta Blocker Meeting (F13Beta) #2
• 2010-03-25 - Test Beta Candidate
• 2010-03-26 - Beta Blocker Meeting (F13Beta) #3
• 2010-03-31 - Beta Go/No-Go Meeting (20:00 EST)

Action items

1. adamw to check-in with wwoods on tooling needs for priv esc. test

IRC transcript

jlaska	#startmeeting Fedora QA meeting	16:00
zodbot	Meeting started Mon Mar 15 16:00:40 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.	16:00
zodbot	Useful Commands: #action #agreed #halp #info #idea #link #topic.	16:00
jlaska	#meetingname fedora-qa	16:00
zodbot	The meeting name has been set to 'fedora-qa'	16:00
jlaska	#topic Gathering ...	16:00
* adamw gathers		16:01
* kparal is here and on eletricity power again		16:01
daumas	there can be only one	16:01
jlaska	kparal: hey, alright	16:01
jlaska	wwoods: is out today	16:02
jlaska	and I'm not expecting lili or rhe to be awake right now	16:03
jlaska	kparal: is jskladan out for the day?	16:03
kparal	jlaska: jskladan was hoping to be online, but it seems he's not here	16:04
jlaska	okay	16:04
jlaska	maxamillion had a conflict too iirc	16:05
jlaska	#topic Previous meeting follow-up	16:05
jlaska	#info maxamillion seeking input from the Mentors group for guidance on mentor responsibilities	16:05
jlaska	not sure of anything new here, anyone else?	16:05
adamw	not heere	16:05
jlaska	I can catch up with maxamillion after	16:05
jlaska	some of these tasks are a bit old ... next up ...	16:06
jlaska	#info continued testing of bug#567346 to further isolate the failure conditions	16:06
jlaska	.bug 567346	16:06
zodbot	jlaska: Bug 567346 gpk-update-viewer does not install updates if there is any dependency issue, and does not correctly report this - https://bugzilla.redhat.com/show_bug.cgi?id=567346	16:06
adamw	that's the update fail bug, right?	16:06
jlaska	yeah, I believe this is the one you updated after Fridays blocker mtg?	16:06
adamw	yeah.	16:06
jlaska	Any objections to dropping this from the list, I think we've got it on the proper radar elsewhere	16:07
adamw	sure	16:07
jlaska	alrighty, next ..	16:08
jlaska	#info jlaska to update AutoQA_PatchProcess to include fedorapeople.org git repo	16:08
jlaska	nothing crazy here, I made a minor update to the AutoQA_PatchProcess wiki page pointing to the instructions for setting up a git repo on fedorapeople.org	16:08
jlaska	AutoQA_PatchProcess#Private_Branch	16:09
jlaska	that's all I have from previous meetings	16:09
jlaska	before we move on, anything else not covered?	16:09
jlaska	alright ... diving into the agenda ...	16:10
jlaska	#topic Fedora 13 test status	16:10
jlaska	this first topic is intended as a check-in on f13 test activities	16:10
jlaska	we've got quite a bit going on at the moment, and the beta is on the horizon	16:10
jlaska	I'm just going to rattle off a few milestones that I'm tracking ...	16:11
jlaska	#info Pre-beta acceptance test - awaiting new anaconda build before sending through AutoQA - https://fedorahosted.org/rel-eng/ticket/3495	16:11
jlaska	next up, we have ...	16:12
jlaska	#info 2010-03-18 - Beta test compose	16:12
jlaska	I'm not seeing tickets yet for the beta composes, I'll catch up with Oxf13 after to make sure I don't fill dups	16:12
jlaska	we also have ...	16:13
jlaska	#info 2010-03-18 - Palimpsest & udisks improvements test day	16:13
jlaska	so quite a bit going on this week, on top of the proposal storm	16:13
* Viking-Ice sneaks inn		16:14
jlaska	I don't have a ticket for the palimpsest test day ... who is coordinating that event?	16:14
jlaska	Viking-Ice: welcome :)	16:14
adamw	i was about to ask the same	16:14
adamw	ot	16:14
adamw	it's on the schedule but i'm not sure who's in charge of that	16:14
adamw	18:02, 16 February 2010 Davidz (Talk | contribs) (3,046 bytes) (Add udisks test day)	16:14
adamw	oh look, it's me!	16:15
adamw	yeah, i have some emails from david about this, from back in february. i'll look after it.	16:15
jlaska	adamw: want to divide and conquer on this one?	16:15
adamw	nah, it's fine, i've got it.	16:15
jlaska	since you had all of webcam last week?	16:15
jlaska	we had a previous palimpsest test day, but I can't find it at the moment ... hopefully that has some useful tests	16:16
adamw	there's stuff on the feature page too.	16:16
jlaska	ah, nm ... I was thinking about DeviceKit (	16:17
jlaska	I'll check-in with Hurry to see if she needs anything for the test compose milestone this week	16:17
adamw	DeviceKit would be right	16:17
adamw	DeviceKit-disks turned into udisks	16:18
jlaska	I thought we had a focus on this already, but then the *Kit name threw me	16:18
adamw	we had HAL, then DeviceKit, then DeviceKit-disks, then udisks	16:18
adamw	easy! =)	16:18
jlaska	kparal: I think we could make a fun flow chart for that progression :)	16:18
jlaska	oh, and last thing on my radar for F13 testing this week	16:19
jlaska	#info 2010-03-19 - F13 Beta Blocker bug review #2	16:19
jlaska	Cranes Maska has quite a few bugs to knock out before that meeting	16:19
jlaska	anything else on the F13 front folks want to share?	16:19
adamw	i wouldn't expect much from that guy	16:19
jlaska	adamw: I set the bar low	16:19
jlaska	alrighty ... moving on ...	16:20
jlaska	#topic proventesters	16:20
jlaska	There has been some discussion on the list in previous weeks regarding the creation of a proventesters group, responsible for providing critpath bodhi feedback for QA	16:21
jlaska	I've seen a few email from Oxf13, adamw and maxamillion, but wanted to give someone a chance to talk about where we are, what's next etc...	16:21
jlaska	is there general agreement on the FAS group name, 'proventesters'?	16:22
adamw	sure	16:22
adamw	brb call of nature	16:22
kparal	just call back :)	16:22
jlaska	heh	16:22
jlaska	so I don't see the group created yet, https://admin.fedoraproject.org/accounts/group/view/proventester ... so that seems like a reasonable starting point	16:23
daumas	jlaska: what will a "proventester" acl give to a person who receives it?	16:24
jlaska	daumas: great question, I'm not sure this process has been fully fleshed out yet	16:24
jlaska	for today, I just wanted to check-in and see where things stood. I know maxamillion has a few threads out to help move things along on this front	16:25
jlaska	perhaps people can lend their thoughts .... /me grabs links ...	16:25
jlaska	http://lists.fedoraproject.org/pipermail/test/2010-March/088980.html	16:25
adamw	basically we're working on maxa's proposal i thought?	16:25
Viking-Ice	what happen with using the already existing QA group?	16:25
adamw	Viking-Ice: http://lists.fedoraproject.org/pipermail/test/2010-March/089065.html	16:26
Viking-Ice	Ah ok good point mentioned there..	16:27
jlaska	adamw: I thought so as well, but I know it's getting a lot of focus and I haven't seen a lot of feedback to maxa's RFC	16:27
jlaska	if nothing else, I'll checking with maxamillion after the meeting and see what he needs to move forward	16:27
adamw	be nice to hear where oxf13 is on it	16:27
jlaska	#Help additional feedback needed, see http://lists.fedoraproject.org/pipermail/test/2010-March/088980.html	16:28
jlaska	#help additional feedback needed, see http://lists.fedoraproject.org/pipermail/test/2010-March/088980.html	16:28
jlaska	adamw: what aspect is Oxf13 working on?	16:29
adamw	i'm not exactly sure, but he was definitely talking about the whole area	16:30
jlaska	alright ... sounds like some data gathering is needed	16:30
jlaska	I'll bug those two later on	16:30
jlaska	okay ... next topic ... back by popular demand	16:30
jlaska	#topic Privilege Escalation test	16:30
jlaska	adamw: I know you've asked about drafting test cases to support the recent priv escalation policy, I wanted to give you some time to talk through it	16:31
adamw	right	16:31
adamw	basically just a reminder that we didn't write the policy for the fun of writing policies =) it was to support privesc testing	16:32
adamw	i believe the next step was to come up with the script to identify privesc-relevant packages	16:32
kparal	should this be automated test for autoqa or manual test, similar to current desktop validation test plan?	16:32
jlaska	kparal: I wonder if these eventually might make good _instrospection_ tests?	16:32
adamw	i'm trying to remember who it was said they would volunteer to do that back at the start, could be wwoods	16:32
adamw	i think there's going to be automated and manual elements to it	16:32
adamw	we can use autoqa to identify packages which have added critpath-relevant stuff, or packages where it changes	16:33
jlaska	wouldn't hurt to start with manual definition	16:33
adamw	and there may be common errors we can catch with autoqa (or may not)	16:33
adamw	but there will also be manual tests we'll want to run on the identified packages	16:33
jlaska	adamw: the best candidate I knew of was for any packages who change/add/remove their polkit definitions?	16:34
adamw	that's part of it, but there are other things to check for too	16:34
adamw	setuid binaries, certain d-bus config files I believe, and consolehelper files are the obvious	16:34
adamw	there's also some lovely outliers out there, like the app I came across the other day which inexplicably chose to use some random thing called 'beesu' which wraps consolehelper	16:35
jlaska	kparal: these seem like good candidates for rpmguard tickets once we have test defined ... since there is a comparison factor here?	16:35
kparal	that's possible	16:35
jlaska	kparal: as you mentioned earlier, I don't want to get too far ahead with tasking out autoqa stuff until we have the current mandatory list (and a place to store/view the results) complete	16:36
jlaska	adamw: is this something you'd like to see in place for Final, or sooner?	16:36
adamw	well the rpmguard stuff will be pretty 'obvious' once we have the tool to generate an overall list in place	16:36
adamw	well it'd be nice to at least have a list of packages for us to eyeball, if nothing else, before final	16:37
jlaska	who can provide that?	16:37
jlaska	adamw: that seems like a reasonable goal ... given the 200 other things everyone has on their plates	16:38
adamw	as I said, i think wwoods said when we started discussing this that he'd be able to come up with a tool	16:39
jlaska	#info interested in a list of packages to apply priv esc tests against for final	16:39
adamw	maybe take an action item for me to talk to him about it? anyone else want to be involved?	16:39
jlaska	#info wwoods previously mentioned producing a tool to provide the list of packages	16:39
jlaska	#action adamw to check-in with wwoods on tooling needs for priv esc. test	16:40
jlaska	adamw: we have the policy in place now to help guide and severity questions around incoming bugs, so that does put us in better shape for F-13 testing than we were in for F-12	16:41
adamw	sure, if something comes up we have a document to point to to say what it should do.	16:41
adamw	but it would definitely be better to have an active testing plan in place rather than just hope for things to float to the surface =)	16:42
jlaska	indeed	16:42
jlaska	alright, anything else on that front ... or just the first step, follow-up w/ wwoods	16:42
adamw	first step is fine for now	16:43
jlaska	eggsellent, thanks adamw	16:43
jlaska	okay, open discussion time ...	16:43
jlaska	#topic Open discussion - <Your topic here>	16:43
jlaska	anything not covered that folks would like to discuss?	16:43
* Viking-Ice nothing from me...		16:44
kparal	well maybe I have a question about the proposals. but we can also discuss it afterwards	16:44
kparal	about the update policy proposals, to be more exact	16:45
jlaska	#topic Update Policy proposals	16:45
jlaska	kparal: take it away	16:45
kparal	alright, just something to be clear on. from the current discussions and proposals, it seems probably that we will have two autoqa checking rounds	16:46
kparal	the first one when accepting packages to updates-testing, the second one when accepting packages to stable updates	16:46
kparal	is that correct?	16:46
jlaska	kparal: that's my understanding	16:47
kparal	because for example I think we want to check dependencies sanity before pushing to updates-testing, not to break our repo	16:47
kparal	OTOH the upgrade path should be checked just when before push to stable updates	16:47
jlaska	right, and then the same to ensure that things are pushed int he right order to 'updates'	16:47
kparal	otherwise the situation may change in between	16:48
jlaska	exactly, for the tests that look for closure in deps and conflicts, it seems like those would need to run during any transition to 'updates-testing' or 'updates'	16:48
kparal	ok, I'm just modifying my proposal again, so I will include these two rounds of autoqa checks. I'm just afraid it will sound even more complicated than the first proposal	16:48
kparal	but the policies are meant to be complex, right? :)	16:49
daumas	kparal: dep checking to stable pushes is loooong overdue	16:49
kparal	daumas: what do you mean?	16:49
jlaska	kparal: given the limited scope of the tests for package acceptance, I don't think we'll get a lot of push back from ensuring that both 'updates' and 'updates-testing' repos are dep solvable	16:50
jlaska	kparal: I'm obviously not an english major, but I'll be happy to look at the wording and propose changes if there is something more concise that can convey the same meaning	16:50
daumas	kparal: pushes get made, users encounter "unable to resolve deps" and complain to the lists. it's been commonplace for a while. it's more rare these days, but most recently happened with nss and nspr	16:50
kparal	alright. I was thinking if we could make the AutoQA process run during the time the package spends in updates-testing. so AutoQA would not be a bottleneck (it may take a few hours)	16:51
kparal	but as I understand it, we can't assume that something that was ok 2 days ago is still ok now	16:51
kparal	e.g. the dependencies	16:51
kparal	so AutoQA will slow it down a little at both sides	16:52
kparal	if I see it wrong please correct me	16:52
jlaska	kparal: one aspect with both policies is that they require autoqa be inproduction	16:53
daumas	kparal: slowing down a few minutes will save days of recoup time.	16:53
jlaska	so the very small test environment we are piloting now won't be representative of the resources available once deployed	16:53
kparal	ok, that's a good answer for me	16:53
jlaska	but honestly, I'd love to have the problem of autoqa being too slow	16:54
jlaska	that would mean all these java packages are behind us :)	16:54
jlaska	#topic Open Discussion - <your topic here>	16:54
jlaska	alright ... anything else for today?	16:54
jlaska	otherwise, I'll call #endmeeting in a few minutes	16:55
adamw	where are you with the autoqa packaging stuff?	16:56
adamw	drowning? need a rope?	16:56
jlaska	adamw: several ropes are being shot out as we speak	16:56
adamw	ah great	16:56
jlaska	adamw: I'll have an email out to java-devel@ and probably devel@ later this week looking for FAD participants	16:56
jlaska	#info adam asked how autoqa packaging was going, jlaska noted that a FAD is in the works, expect more later this week	16:57
jlaska	alright gang ... thanks for your time	16:57
jlaska	as usual, I'll send minutes to the list	16:57
jlaska	#endmeeting	16:57

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!