Revision as of 19:13, 30 November 2009

Attendees

Agenda

Previous meeting follow-up

adamw - initiate security policy discussion on fedora-{devel,security}-list (see http://spot.livejournal.com/312216.html)
jlaska to send request for retrospective feedback to fedora-test-list@

Enhancing Release Criteria

John Poelstra has been thinking about enhancements to the current Fedora release criteria and has organized his thoughts into several wiki pages, starting with Fedora_Release_Criteria. There aren't intentions that this list will be the end-all be-all of checklists for the list. I know we've all experienced some of the subjective nature of identifying blocker bugs and assessing release impact.

Please take a few moments to review the criteria, along with the following pages, and offer your suggestions/corrections.

Fedora 13 release criteria

General FAQ around escalating blocker bugs - Blocker_Bug_FAQ

User:poelstra joined the meeting and recommended reading the email announcement first. John also indicated he was hoping this would set a better stage for more info that might come from the target audience discussion. The plan was to host healthy discussion on the mailing list until FUDCon. Then, at FUDCon, hammer out the dents and formalize something we can use for Fedora 13. John asked the team if this was realistic. Adamw and jlaska felt it was.

Wwoods noted that the original release criteria was started by writing down whatever unwritten common-sense tests and policies we already had, and maybe a couple "it'd be nice if.." ones. John thanked Will for starting the release criteria process, as many of the points raised in the original page are included in the new release-specific pages.

Security Test Plan

Spot posted some great points around non-root user security expectations in his blog last week (see http://spot.livejournal.com/312216.html). This seems to me like an interesting project worth pursuing for Fedora 13.

The team discussed at length, highlights include:

Spot updated his blog with the latest feedback from his blog -- see http://spot.livejournal.com/312216.html
part#1 - Should involve defining a security policy ... instead of making one up
- This policy may take into account a method for each spin SIG to add spin-specific security policy.
part#2 - The QA team can support a security policy by creating test documentation (plans/cases) and providing test results

AdamW agreed to take the first step by initiating discussion with fedora-devel-list and fedora-security-list to begin the process of reaching consensus on a security policy.

Fedora 12 QA retrospective

See announcement - https://www.redhat.com/archives/fedora-test-list/2009-November/msg01126.html
See wiki page - Fedora_12_QA_Retrospective

AutoQA update

wwoods updates

autoqa-0.3 merged into the master branch (see http://git.fedorahosted.org/git/?p=autoqa.git;a=commit;h=a27a03f527a76af24bac46dd3872efc9fd4e3984). This branch adds:

a new autoqa python library
which includes, shared repoinfo code for the watcher scripts and utility functions/classes for tests
the new post-koji-build hook. Which is still fairly experimental, but we have it running a simple 'rpmlint' test on every new build that comes out of koji

kparal updates

Kamil's plan for this week includes working on integration of rpmguard into autoqa. Wwoods noted that they'd need to figure out what to do with the output of the test (mail it to package owners/autoqa-results?) and what to do when there's a change that should block the package. But that can wait until after the test is working.

Misc

Support for autoqa --local - https://fedorahosted.org/autoqa/ticket/52 - DONE
Convert israwhidebroken to WSGI - https://fedorahosted.org/autoqa/ticket/91 - POST - awaiting review
Use cases - https://fedoraproject.org/wiki/AutoQA_Use_Cases - INPROGRESS

Open discussion - <Your topic here>

Upcoming QA events

NA

Action items

IRC transcript

jlaska	#startmeeting Fedora QA Meeting	16:00
zodbot	Meeting started Mon Nov 30 16:00:20 2009 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.	16:00
zodbot	Useful Commands: #action #agreed #halp #info #idea #link #topic.	16:00
jlaska	#meetingname qa	16:00
zodbot	The meeting name has been set to 'qa'	16:00
jlaska	#topic Gathering in the lobby	16:00
adamw	yo	16:00
* kparal		16:01
* tk009 is kinda here		16:01
jlaska	adamw: kparal: tk009 Howdy folks	16:01
tk009	good morning	16:01
* jlaska notes ... folks in the US might still be a bit sleepy from consuming too much turkey		16:02
jlaska	wwoods should be around, might just be gathering a few breadths after a near-miss of a system failure	16:03
* wwoods is indeed around		16:04
jlaska	anyone else ... Viking-Ice: poelcat: Oxf13?	16:04
* poelcat here		16:04
jlaska	poelcat: greetings	16:05
poelcat	hi, thanks for the reminder	16:05
jlaska	okay, let's dive in ... don't want to go too long (for a change)	16:05
jlaska	#link https://www.redhat.com/archives/fedora-test-list/2009-November/msg01223.html	16:05
jlaska	^^^ the proposed agenda for today	16:05
jlaska	#topic Previous meeting follow-up	16:05
jlaska	* adamw - initiate security policy discussion on fedora-{devel,security}-list (see http://spot.livejournal.com/312216.html)	16:05
jlaska	that's a [X], that's for kicking that off adamw	16:06
adamw	so, i did it	16:06
jlaska	I've got a spot in the agenda to talk about progress there	16:06
adamw	ok	16:06
jlaska	anything else high-level to discuss there?	16:06
jlaska	* jlaska to send request for retrospective feedback to fedora-test-list@	16:07
jlaska	same for me ... this was sent, folks have contributed already (thank you) ... and will do a status updates later in the meeting	16:07
jlaska	that's all I have from last week's action items	16:07
jlaska	anything I missed?	16:07
jlaska	okay ... let's move on then	16:08
jlaska	#topic Enhancing release criteria	16:08
jlaska	poelcat introduced this topic and we discussed it from a high-level last week	16:09
poelcat	thanks to everyone who gave feedback on the list and talk pages	16:09
* jlaska notes ... poelcat beat me too it :)		16:09
poelcat	what do you think of next step of taking all that feedback and working it into the pages	16:10
poelcat	and then send another reminder to the list by say tomorrow	16:10
adamw	i think it would be a great job for someone whose irc nick is poelcat ;)	16:10
poelcat	and then do an in person review/finalize this weekend!	16:10
poelcat	at FUDCon :)	16:10
jlaska	I've been adjusting the pages for the Talk feedback so far	16:10
poelcat	jlaska: excellent	16:11
jlaska	A hackfest session @ FUDCon?	16:11
poelcat	i'll touch base w/ you after the meeting to see if there are any holes	16:11
poelcat	jlaska: that was my thought	16:11
jlaska	I'm still unclear on how best to account for what adamw raised	16:11
poelcat	seems too specialized for barcamp	16:11
poelcat	jlaska: what is that?	16:12
adamw	which bit?	16:12
jlaska	the impact on severity of hardware and local configuration issues	16:12
jlaska	we all know it exists, but unclear on how best to put that into words	16:12
adamw	i can write up a paragraph for that if you like	16:12
jlaska	no objections here ... it's better than the empty string I've got	16:13
* poelcat wonders if there is still a possiblity of some quantificapability?		16:13
jlaska	infrastructuralization?	16:13
jlaska	:)	16:13
poelcat	e.g. "5 or more systems" ?	16:13
* Viking-Ice joins late inn..		16:13
jlaska	Viking-Ice: welcome!	16:13
adamw	poelcat: it's hard to do even that	16:13
poelcat	adamw: true	16:14
jlaska	we definitely want bugs filed ... but perhaps this data is used to facilitate the blocker bug decision	16:14
* poelcat was thinking of the easy cases		16:14
adamw	poelcat: you could find five systems that hit virtually any kernel bug, from kerneloops.org	16:14
poelcat	we can move on	16:14
jlaska	yeah sorry, rat hole	16:14
adamw	on the contrary, there are some issues you might be hard-pressed to get five reporters for in beta stage but which many would hit in final	16:15
adamw	so, yeah, i'd prefer to keep it general.	16:15
jlaska	#info next step ... of taking all that feedback and working it into the pages	16:15
jlaska	#info send another reminder to the list by say tomorrow	16:15
adamw	can you #action me?	16:15
jlaska	#info hackfest session at FUDCon to finalize	16:15
jlaska	#action adamw to offer some guidance on how to handle hardware/local_configuration specific bugs	16:16
jlaska	poelcat: thanks for the update, anything else?	16:16
poelcat	jlaska: that's all	16:16
jlaska	#topic Security Policy/Test_Plan	16:17
jlaska	adamw: do you have a few minutes to talk about where that stands, issues/roadblocks etc... ?	16:17
jlaska	#link https://www.redhat.com/archives/fedora-devel-list/2009-November/msg01745.html	16:17
adamw	well	16:18
adamw	the discussion has done a classic fedora peter-out	16:18
adamw	it seems like there's vague consensus that we should have package defaults that are 'very secure' and then per-spin customization for less security (if desired)	16:19
adamw	though that would seem to have lots of holes in it as an idea	16:19
adamw	no-one has really stepped up and said 'yeah there's a plan and we're doing it here'	16:19
adamw	so i may need to give it another poke	16:19
poelcat	adamw: is FESCo tracking this issue?	16:19
adamw	poelcat: I don't know.	16:20
* poelcat would hope/think they have a ticket open and are discussing		16:20
adamw	we could ask...	16:20
adamw	nirik: ping?	16:20
Oxf13	hey, I'm here, just running late.	16:20
Oxf13	kid decided to wake up at 5am and have a meltdown.	16:20
nirik	adamw: whats up?	16:21
jlaska	Oxf13: welcome!	16:21
adamw	nirik: is FESco tracking the security policy question?	16:21
adamw	nirik: see discussion about (since xx:17:17)	16:21
adamw	s/about/above/	16:21
skvidal	adamw: I wrote a draft of the significant change policy	16:21
skvidal	and I sent it to the fesco mailing list last week	16:21
* nirik was just typing that...		16:21
skvidal	I've, as yet, received no comment At all	16:22
skvidal	which makes me sad	16:22
skvidal	I can send it to fedora-devel-list if that would help conversation	16:22
nirik	skvidal: sorry, I was busy friday and didn't feel like answering over the weekend. It's in my mailbox to reply to today...	16:22
skvidal	nirik: so folks did GET it	16:22
skvidal	that's good	16:22
nirik	yes, I got it. Just haven't had a chance to reply yet	16:23
jlaska	#info there's vague consensus that we should have package defaults that are 'very secure' and then per-spin customization for less security (if desired)	16:23
skvidal	b/c I couldn't tell if it had been held for approval or something	16:23
jlaska	#info no-one has really stepped up and said 'yeah there's a plan and we're doing it here'	16:23
nirik	with thanksgiving and the long weekend, I suspect many people were away from email.	16:23
adamw	skvidal: do you think the 'significant change policy' is going to be enough to act as a 'security policy'?	16:23
Viking-Ice	One question is there any security certification we can get on Fedora ( some one mentioned something NIST/DISA thingy ) if so is that something we would like to have ( for example a security certified workstation/server spin )	16:23
jlaska	#info skvidal drafted a change policy and sent to fesco mailing list last week	16:23
skvidal	adamw: no - but it should help us not get caught out in the cold on changes	16:23
nirik	We may want to also look at a 'secure by default, spins/etc can relax for their needs' policy... (since we don't have such a thing right now)	16:24
adamw	that's kinda what I thought. i still feel like there's a need for security policy (policies) / packaging guidelines. so fesco doesn't have anything going on that front?	16:24
nirik	adamw: not currently, but concrete proposals welcome.	16:25
adamw	nirik: that involves defining what 'secure' means, also runs into the 'desktop spin' border issue, and at least one security team member has posted to the thread that he isn't happy with the idea of spin SIGs having complete control over security policy in their spins.	16:25
adamw	nirik: OK, thanks.	16:25
* nirik nods.		16:26
Viking-Ice	Well I do belive we should allow spins to have complete control over their own security	16:26
adamw	Viking-Ice: we don't really need to debate it here	16:26
adamw	Viking-Ice: just flagging up the issues	16:26
adamw	it's not qa group's job to decide the answers	16:26
Viking-Ice	They just need to document how the deviate from the uber secure policy ;)	16:26
adamw	skvidal: um, where is the fesco mailing list? is it private? it doesn't seem to be listed on the fesco wiki page	16:27
Oxf13	if anything, it'd be QAs job to ensure the security policy as defined by the project and/or the spin matches reality	16:27
adamw	Oxf13: right.	16:27
skvidal	adamw: 'fedora-extras-steering'	16:27
skvidal	adamw: it's private, I believe	16:27
adamw	Oxf13: that's where we came into this issue: if we want to do testing we need a policy to test against.	16:27
adamw	skvidal: ah, k.	16:27
skvidal	adamw: it's mostly so we can talk trash about you. :)	16:27
adamw	skvidal: that's what i figured =)	16:27
jlaska	okay, so help me capture next steps here	16:27
jlaska	is profit one of them yet?	16:28
adamw	i guess i should file a ticket with fesco	16:28
adamw	and profit is ALWAYS one of the steps	16:28
nirik	yeah, a ticket with a proposed policy would be great.	16:28
nirik	or a post to devel with it, wait for flames to subside, then a ticket. ;)	16:28
adamw	i'm not sure i'm the guy to propose a policy	16:29
poelcat	nirik: fesco is expecting someone else to propose a policy?	16:29
adamw	given that i have precisely zero security qualifications	16:29
nirik	poelcat: expecting? probibly not, just hoping. ;)	16:29
adamw	i was thinking more along the lines of a ticket asking fesco to look into the issue, flagging up the problematic areas	16:29
jlaska	adamw: nirik: and the ticket is to just get this on FESCO's radar?	16:29
* nirik can try and do something if nothing appears soon.		16:29
nirik	adamw: thats fine too.	16:29
adamw	if it'd be preferred, i could ask the security team to work with me to come up with a proposed policy	16:29
Viking-Ice	+1	16:30
* poelcat hoping to say this is the nicest way, but would expect that creating/reviewing/make sure a security policy happens is FESCo's job		16:30
* poelcat realizes we are off topic for this meeting		16:30
Viking-Ice	I do believe having the sec xperts on board and preferable defining the sec policy is the way to go.. .	16:31
jlaska	#info next step ... file a FESCO ticket asking fesco to look into the issue and help us move towards a security policy	16:31
nirik	poelcat: sure, agreed.	16:31
Oxf13	poelcat: while that is true, you don't have to be in FESCo to create the policy or any proposal.	16:31
adamw	just #action me to do a fesco ticket and i'll figure it out	16:31
Oxf13	poelcat: in fact, a subject matter expert may be a better choice to create such a policy, rather than the members of FESCo	16:32
* adamw is talking to security team now		16:32
jlaska	#action adamw will reach out to FESCO for guidance on defining a security policy	16:32
poelcat	Oxf13: yep, that is part of "make sure a security policy happens"	16:33
jlaska	adamw: okay ... anything else I missed or need to capture?	16:33
Oxf13	poelcat: that also assumes that FESCo or the Fedora Board has determined that a security policy is even necessary.	16:34
poelcat	we're off topic, let's move on :)	16:34
adamw	jlaska: i think that's okay.	16:34
jlaska	adamw: alright, thank you	16:34
jlaska	#topic F-12 QA retrospective	16:35
jlaska	#link https://www.redhat.com/archives/fedora-test-list/2009-November/msg01126.html	16:35
jlaska	lots of good feedback on the wiki and mailing list already, so thanks to all who contributed so far	16:35
jlaska	I'm still migrating mailing list feedback into the wiki page, I expect to wrap that up today	16:35
jlaska	#link https://fedoraproject.org/wiki/Fedora_12_QA_Retrospective	16:36
jlaska	As for next steps ... I plan to organize the feedback into related groups to make it easier to see trends/problem_areas	16:37
jlaska	I'd like to then see some discussion around what the group wants to focus on for F-13 testing	16:37
jlaska	I'm still uncertain on the most effective forum for that ... I'm guessing a mix of mailing list and a FUDCon break-out	16:38
jlaska	this shouldn't be anything new ... we informally did this for F-12	16:38
jlaska	all I'm trying to do now is make this process a bit more repeatable+transparent ... and hopefully help interested participants take part in our objectives for F-13	16:39
jlaska	#info Next steps ... I plan to organize the feedback into related groups to make it easier to see trends/problem_areas	16:39
jlaska	anyone else have questions or concerns on this topic?	16:40
adamw	nothing really	16:41
Oxf13	nope	16:41
* Viking-Ice nothing from me..		16:41
jlaska	alrighty ... let's move on then	16:41
jlaska	#topic AutoQA Updates	16:41
jlaska	Just our usual check-in to see how things are progressing on the wwoods and kparal autoqa front	16:42
jlaska	who wants to go first?	16:42
kparal	ok, a few words from me :)	16:42
wwoods	er, I'll go real quick	16:42
kparal	heh	16:42
wwoods	or.. yes	16:42
wwoods	kparal: you go first	16:42
kparal	alright	16:43
jlaska	hehe, sorry guys ... I'll pick one of you first to avoid confusion next time	16:43
kparal	there are a few patches in the autoqa mailing from me, maybe you have seen them	16:43
kparal	the purpose is to make test development even easier	16:43
wwoods	kparal: ooh, I didn't see the optparse patch for the watchers	16:44
kparal	so all the watchers should have now (after accepting the patches) --help and --dry-run standardized	16:44
* wwoods was out of the office and off the VPN Nov. 25 until now, still catching up		16:44
kparal	so now it could improve the expectations that people have when trying out the code	16:44
kparal	I will start documenting the 'Getting started' stuff after it's in master	16:45
jlaska	#info kparal has a few autoqa patches out for review - adding --help and --dry-run options to all watchers	16:45
kparal	and I also worked a little bit on integrating rpmguard into autoqa, but not finished yet	16:45
jlaska	#info kparal plans to contribute to the 'Getting started' use case once changes are in	16:46
* jlaska going crazy with meetbot tags ... sorry for annoyance		16:46
kparal	wwoods, you may go on now :)	16:46
wwoods	heh - thanks, kparal	16:47
jlaska	#info kparal some progress on integrating rpmguard, more work remains	16:47
wwoods	So last week I adapted kparal's patch for the autoqa harness to add the --local flag	16:47
wwoods	so tests can be launched on your local system, to help with test development	16:47
wwoods	it also supports the --dry-run flag, like the watchers	16:48
kparal	already tried that, works perfect	16:48
wwoods	kparal: great!	16:48
wwoods	let's see - there were some fixes to the watchers	16:49
jlaska	#info Wwoods accepted kparal's local autoqa harness patch ... tests can now be launched on your local system to facilitate test development/integration	16:49
wwoods	they should be running autoqa only once for each repo/build that's updated, regardless of which (or how many) arches the build/tree/repo is available for	16:49
wwoods	this is going to be important when we start working with more 'noarch' tests - tests like rpmlint, which don't need to run on the same arch as the target package/tree	16:50
wwoods	also watch-repos.py was running tests too often, that was fixed	16:51
wwoods	and I updated repoinfo to reflect the fact that there's no ppc rawhide anymore	16:51
* jlaska makes a note to ensure we're tracking the documentation needed for autotest system tagging		16:51
wwoods	with the new repoinfo library that's just a simple config file change, so that's nice	16:51
wwoods	in preparation for FUDCon I'm planning to write up some notes on AutoQA for Fedora developers	16:52
* jlaska was impressed with the repoinfo config file		16:52
wwoods	and hopefully I'll be helping kparal get rpmguard running and sending emails and stuff	16:52
wwoods	that's all from me.	16:53
jlaska	wwoods: great stuff	16:53
* jlaska info's ...		16:53
jlaska	#info wwoods improved support for running 'noarch' tests - tests like rpmlint, which don't need to run on the same arch as the target package/tree	16:53
jlaska	#info wwoods updated repoinfo to reflect the fact that there's no ppc rawhide anymore	16:54
jlaska	#info wwoods plans ... some FUDCon prep work and helping kparal integrate rpmguard	16:54
jlaska	I think that gets it ... shout if I've missed anything	16:54
wwoods	to be fair, the current support for 'noarch' tests is kind of a gross hack	16:54
jlaska	we like hacks	16:55
wwoods	but I laid some of the groundwork to make it work sanely later	16:55
* jlaska makes a note to head to cafepress after meeting :)		16:55
skvidal	wwoods: repoinfo is in autoqa?	16:55
wwoods	skvidal: yes	16:55
skvidal	wwoods: thanks	16:55
jlaska	Just a few updates on some of my autoqa action items ...	16:56
jlaska	#info thanks to help from abadger1999, autoqa-israwhidebroken is now packaged. I sent a link to the branch to autoqa-devel. If folks are happy, I can merge that into master	16:56
jlaska	having this packaged highlighted a few next steps for having this service run as http://admin.fedoraproject.org/israwhidebroken	16:57
jlaska	I don't think I'll be able to knock those out before FUDCon, so might look for guidance from abadger1999 and/or mmcgrath if they have the time	16:57
jlaska	wwoods: kparal: anything else on the autoqa front?	16:58
kparal	not from me	16:58
jlaska	kparal: okay ... oh, wwoods said he was good earlier ... alright, next up ...	16:59
jlaska	#topic Open Discussion - <your topic here>	16:59
adamw	FINALLY time for the week's most important topic	16:59
jlaska	any topics not yet mentioned people would like to discuss?	16:59
jlaska	adamw: hit it!	16:59
adamw	i move that glogg be adopted as the new official drink of QA	16:59
adamw	http://loupgaroublond.blogspot.com/2009/11/glogg.html	16:59
adamw	it involves wine, port and whiskey, and has an awesome name. i see no drawbacks.	16:59
jlaska	#topic Open Discussion - Glögg as the official drink of QA	17:00
wwoods	glogg!!	17:00
wwoods	I have a bottle of that in my house right now	17:00
jlaska	adamw: if we drink too much, side affects?	17:00
Oxf13	adamw: I was thinking more 'Magic Hobo Gravy' http://hijinksensue.com/2009/11/27/the-special-sauce/	17:00
jlaska	"Best code I ever wrote!"	17:00
kparal	I want pictures! :)	17:00
adamw	i do not understand what you mean by 'too much'	17:00
adamw	this seems a nonsensical concept	17:00
jlaska	the first images.google.com result for grogg is not what I'd expect	17:01
jlaska	adamw: :)	17:01
jlaska	http://www.mostphotos.com/preview/150009/grogg-drink-ice-glas-red.jpg ?	17:01
adamw	iced grogg?	17:01
jlaska	http://www.drunkenblog.com/drunkenblog-archives/i/oh_glogg_youdevil.jpg	17:01
Oxf13	jlaska: "grogg" vs "glogg"	17:01
wwoods	yeah I think maybe Glögg and grog(g) are different things	17:01
adamw	http://www.acatinthekitchen.com/photo/advent/glogg.jpg	17:01
jlaska	indeed they are ... never trust "Maybe you meant grogg" :)	17:02
jlaska	kparal: I've seen this a lot http://www.saturnus.se/images/produkter/glogg_export.jpg	17:02
jlaska	okay ... anything else on the radar we need to discuss?	17:02
kparal	never seen that in czech republic	17:02
Oxf13	yeah, who is going to be on the bus?	17:03
jlaska	kparal: You've got some of the best stuff on earth right in your back yard :)	17:03
jlaska	#topic Open Discussion - FUDCon travel plans	17:03
jlaska	#topic Open Discussion - FUDCon travel plans	17:04
jlaska	nirik: thanks!	17:04
wwoods	jlaska and I will be flying into Toronto on Friday. Remember yer passports, USians.	17:04
* jlaska wonders when the next EMEA FUDCon is		17:05
jlaska	okay ... I'll close out the meeting in 2 minutes unless any urgent items come up	17:06
jlaska	alright folks ... I think we can close it out	17:08
jlaska	thanks for your time	17:08
jlaska	As always, minutes will be sent to the list	17:08
jlaska	#endmeeting	17:08

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!

Search

QA/Meetings/20091130: Difference between revisions