From Fedora Project Wiki

< QA‎ | Meetings
Revision as of 19:14, 30 November 2009 by Jlaska (talk | contribs) (Update attendees)

Attendees

People present (lines said)

  • jlaska (115)
  • adamw (53)
  • wwoods (27)
  • poelcat (25)
  • kparal (15)
  • nirik (14)
  • skvidal (14)
  • Oxf13 (10)
  • Viking-Ice (7)
  • zodbot (3)
  • tk009 (2)

Regrets:

Agenda

Previous meeting follow-up

Enhancing Release Criteria

John Poelstra has been thinking about enhancements to the current Fedora release criteria and has organized his thoughts into several wiki pages, starting with Fedora_Release_Criteria. There aren't intentions that this list will be the end-all be-all of checklists for the list. I know we've all experienced some of the subjective nature of identifying blocker bugs and assessing release impact.

Please take a few moments to review the criteria, along with the following pages, and offer your suggestions/corrections.

Fedora 13 release criteria

General FAQ around escalating blocker bugs - Blocker_Bug_FAQ

User:poelstra joined the meeting and recommended reading the email announcement first. John also indicated he was hoping this would set a better stage for more info that might come from the target audience discussion. The plan was to host healthy discussion on the mailing list until FUDCon. Then, at FUDCon, hammer out the dents and formalize something we can use for Fedora 13. John asked the team if this was realistic. Adamw and jlaska felt it was.

Wwoods noted that the original release criteria was started by writing down whatever unwritten common-sense tests and policies we already had, and maybe a couple "it'd be nice if.." ones. John thanked Will for starting the release criteria process, as many of the points raised in the original page are included in the new release-specific pages.

Security Test Plan

Spot posted some great points around non-root user security expectations in his blog last week (see http://spot.livejournal.com/312216.html). This seems to me like an interesting project worth pursuing for Fedora 13.

The team discussed at length, highlights include:

  • Spot updated his blog with the latest feedback from his blog -- see http://spot.livejournal.com/312216.html
  • part#1 - Should involve defining a security policy ... instead of making one up
    • This policy may take into account a method for each spin SIG to add spin-specific security policy.
  • part#2 - The QA team can support a security policy by creating test documentation (plans/cases) and providing test results

AdamW agreed to take the first step by initiating discussion with fedora-devel-list and fedora-security-list to begin the process of reaching consensus on a security policy.

Fedora 12 QA retrospective


AutoQA update

wwoods updates

autoqa-0.3 merged into the master branch (see http://git.fedorahosted.org/git/?p=autoqa.git;a=commit;h=a27a03f527a76af24bac46dd3872efc9fd4e3984). This branch adds:

  • a new autoqa python library
  • which includes, shared repoinfo code for the watcher scripts and utility functions/classes for tests
  • the new post-koji-build hook. Which is still fairly experimental, but we have it running a simple 'rpmlint' test on every new build that comes out of koji

kparal updates

Kamil's plan for this week includes working on integration of rpmguard into autoqa. Wwoods noted that they'd need to figure out what to do with the output of the test (mail it to package owners/autoqa-results?) and what to do when there's a change that should block the package. But that can wait until after the test is working.

Misc

Open discussion - <Your topic here>

Upcoming QA events

  • NA

Action items

IRC transcript

jlaska #startmeeting Fedora QA Meeting 16:00
zodbot Meeting started Mon Nov 30 16:00:20 2009 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00
jlaska #meetingname qa 16:00
zodbot The meeting name has been set to 'qa' 16:00
jlaska #topic Gathering in the lobby 16:00
adamw yo 16:00
* kparal 16:01
* tk009 is kinda here 16:01
jlaska adamw: kparal: tk009 Howdy folks 16:01
tk009 good morning 16:01
* jlaska notes ... folks in the US might still be a bit sleepy from consuming too much turkey 16:02
jlaska wwoods should be around, might just be gathering a few breadths after a near-miss of a system failure 16:03
* wwoods is indeed around 16:04
jlaska anyone else ... Viking-Ice: poelcat: Oxf13? 16:04
* poelcat here 16:04
jlaska poelcat: greetings 16:05
poelcat hi, thanks for the reminder 16:05
jlaska okay, let's dive in ... don't want to go too long (for a change) 16:05
jlaska #link https://www.redhat.com/archives/fedora-test-list/2009-November/msg01223.html 16:05
jlaska ^^^ the proposed agenda for today 16:05
jlaska #topic Previous meeting follow-up 16:05
jlaska * adamw - initiate security policy discussion on fedora-{devel,security}-list (see http://spot.livejournal.com/312216.html) 16:05
jlaska that's a [X], that's for kicking that off adamw 16:06
adamw so, i did it 16:06
jlaska I've got a spot in the agenda to talk about progress there 16:06
adamw ok 16:06
jlaska anything else high-level to discuss there? 16:06
jlaska * jlaska to send request for retrospective feedback to fedora-test-list@ 16:07
jlaska same for me ... this was sent, folks have contributed already (thank you) ... and will do a status updates later in the meeting 16:07
jlaska that's all I have from last week's action items 16:07
jlaska anything I missed? 16:07
jlaska okay ... let's move on then 16:08
jlaska #topic Enhancing release criteria 16:08
jlaska poelcat introduced this topic and we discussed it from a high-level last week 16:09
poelcat thanks to everyone who gave feedback on the list and talk pages 16:09
* jlaska notes ... poelcat beat me too it :) 16:09
poelcat what do you think of next step of taking all that feedback and working it into the pages 16:10
poelcat and then send another reminder to the list by say tomorrow 16:10
adamw i think it would be a great job for someone whose irc nick is poelcat ;) 16:10
poelcat and then do an in person review/finalize this weekend! 16:10
poelcat at FUDCon :) 16:10
jlaska I've been adjusting the pages for the Talk feedback so far 16:10
poelcat jlaska: excellent 16:11
jlaska A hackfest session @ FUDCon? 16:11
poelcat i'll touch base w/ you after the meeting to see if there are any holes 16:11
poelcat jlaska: that was my thought 16:11
jlaska I'm still unclear on how best to account for what adamw raised 16:11
poelcat seems too specialized for barcamp 16:11
poelcat jlaska: what is that? 16:12
adamw which bit? 16:12
jlaska the impact on severity of hardware and local configuration issues 16:12
jlaska we all know it exists, but unclear on how best to put that into words 16:12
adamw i can write up a paragraph for that if you like 16:12
jlaska no objections here ... it's better than the empty string I've got 16:13
* poelcat wonders if there is still a possiblity of some quantificapability? 16:13
jlaska infrastructuralization? 16:13
jlaska :) 16:13
poelcat e.g. "5 or more systems" ? 16:13
* Viking-Ice joins late inn.. 16:13
jlaska Viking-Ice: welcome! 16:13
adamw poelcat: it's hard to do even that 16:13
poelcat adamw: true 16:14
jlaska we definitely want bugs filed ... but perhaps this data is used to facilitate the blocker bug decision 16:14
* poelcat was thinking of the easy cases 16:14
adamw poelcat: you could find five systems that hit virtually any kernel bug, from kerneloops.org 16:14
poelcat we can move on 16:14
jlaska yeah sorry, rat hole 16:14
adamw on the contrary, there are some issues you might be hard-pressed to get five reporters for in beta stage but which many would hit in final 16:15
adamw so, yeah, i'd prefer to keep it general. 16:15
jlaska #info next step ... of taking all that feedback and working it into the pages 16:15
jlaska #info send another reminder to the list by say tomorrow 16:15
adamw can you #action me? 16:15
jlaska #info hackfest session at FUDCon to finalize 16:15
jlaska #action adamw to offer some guidance on how to handle hardware/local_configuration specific bugs 16:16
jlaska poelcat: thanks for the update, anything else? 16:16
poelcat jlaska: that's all 16:16
jlaska #topic Security Policy/Test_Plan 16:17
jlaska adamw: do you have a few minutes to talk about where that stands, issues/roadblocks etc... ? 16:17
jlaska #link https://www.redhat.com/archives/fedora-devel-list/2009-November/msg01745.html 16:17
adamw well 16:18
adamw the discussion has done a classic fedora peter-out 16:18
adamw it seems like there's vague consensus that we should have package defaults that are 'very secure' and then per-spin customization for less security (if desired) 16:19
adamw though that would seem to have lots of holes in it as an idea 16:19
adamw no-one has really stepped up and said 'yeah there's a plan and we're doing it here' 16:19
adamw so i may need to give it another poke 16:19
poelcat adamw: is FESCo tracking this issue? 16:19
adamw poelcat: I don't know. 16:20
* poelcat would hope/think they have a ticket open and are discussing 16:20
adamw we could ask... 16:20
adamw nirik: ping? 16:20
Oxf13 hey, I'm here, just running late. 16:20
Oxf13 kid decided to wake up at 5am and have a meltdown. 16:20
nirik adamw: whats up? 16:21
jlaska Oxf13: welcome! 16:21
adamw nirik: is FESco tracking the security policy question? 16:21
adamw nirik: see discussion about (since xx:17:17) 16:21
adamw s/about/above/ 16:21
skvidal adamw: I wrote a draft of the significant change policy 16:21
skvidal and I sent it to the fesco mailing list last week 16:21
* nirik was just typing that... 16:21
skvidal I've, as yet, received no comment At all 16:22
skvidal which makes me sad 16:22
skvidal I can send it to fedora-devel-list if that would help conversation 16:22
nirik skvidal: sorry, I was busy friday and didn't feel like answering over the weekend. It's in my mailbox to reply to today... 16:22
skvidal nirik: so folks did GET it 16:22
skvidal that's good 16:22
nirik yes, I got it. Just haven't had a chance to reply yet 16:23
jlaska #info there's vague consensus that we should have package defaults that are 'very secure' and then per-spin customization for less security (if desired) 16:23
skvidal b/c I couldn't tell if it had been held for approval or something 16:23
jlaska #info no-one has really stepped up and said 'yeah there's a plan and we're doing it here' 16:23
nirik with thanksgiving and the long weekend, I suspect many people were away from email. 16:23
adamw skvidal: do you think the 'significant change policy' is going to be enough to act as a 'security policy'? 16:23
Viking-Ice One question is there any security certification we can get on Fedora ( some one mentioned something NIST/DISA thingy ) if so is that something we would like to have ( for example a security certified workstation/server spin ) 16:23
jlaska #info skvidal drafted a change policy and sent to fesco mailing list last week 16:23
skvidal adamw: no - but it should help us not get caught out in the cold on changes 16:23
nirik We may want to also look at a 'secure by default, spins/etc can relax for their needs' policy... (since we don't have such a thing right now) 16:24
adamw that's kinda what I thought. i still feel like there's a need for security policy (policies) / packaging guidelines. so fesco doesn't have anything going on that front? 16:24
nirik adamw: not currently, but concrete proposals welcome. 16:25
adamw nirik: that involves defining what 'secure' means, also runs into the 'desktop spin' border issue, and at least one security team member has posted to the thread that he isn't happy with the idea of spin SIGs having complete control over security policy in their spins. 16:25
adamw nirik: OK, thanks. 16:25
* nirik nods. 16:26
Viking-Ice Well I do belive we should allow spins to have complete control over their own security 16:26
adamw Viking-Ice: we don't really need to debate it here 16:26
adamw Viking-Ice: just flagging up the issues 16:26
adamw it's not qa group's job to decide the answers 16:26
Viking-Ice They just need to document how the deviate from the uber secure policy ;) 16:26
adamw skvidal: um, where is the fesco mailing list? is it private? it doesn't seem to be listed on the fesco wiki page 16:27
Oxf13 if anything, it'd be QAs job to ensure the security policy as defined by the project and/or the spin matches reality 16:27
adamw Oxf13: right. 16:27
skvidal adamw: 'fedora-extras-steering' 16:27
skvidal adamw: it's private, I believe 16:27
adamw Oxf13: that's where we came into this issue: if we want to do testing we need a policy to test against. 16:27
adamw skvidal: ah, k. 16:27
skvidal adamw: it's mostly so we can talk trash about you. :) 16:27
adamw skvidal: that's what i figured =) 16:27
jlaska okay, so help me capture next steps here 16:27
jlaska is profit one of them yet? 16:28
adamw i guess i should file a ticket with fesco 16:28
adamw and profit is ALWAYS one of the steps 16:28
nirik yeah, a ticket with a proposed policy would be great. 16:28
nirik or a post to devel with it, wait for flames to subside, then a ticket. ;) 16:28
adamw i'm not sure i'm the guy to propose a policy 16:29
poelcat nirik: fesco is expecting someone else to propose a policy? 16:29
adamw given that i have precisely zero security qualifications 16:29
nirik poelcat: expecting? probibly not, just hoping. ;) 16:29
adamw i was thinking more along the lines of a ticket asking fesco to look into the issue, flagging up the problematic areas 16:29
jlaska adamw: nirik: and the ticket is to just get this on FESCO's radar? 16:29
* nirik can try and do something if nothing appears soon. 16:29
nirik adamw: thats fine too. 16:29
adamw if it'd be preferred, i could ask the security team to work with me to come up with a proposed policy 16:29
Viking-Ice +1 16:30
* poelcat hoping to say this is the nicest way, but would expect that creating/reviewing/make sure a security policy happens is FESCo's job 16:30
* poelcat realizes we are off topic for this meeting 16:30
Viking-Ice I do believe having the sec xperts on board and preferable defining the sec policy is the way to go.. . 16:31
jlaska #info next step ... file a FESCO ticket asking fesco to look into the issue and help us move towards a security policy 16:31
nirik poelcat: sure, agreed. 16:31
Oxf13 poelcat: while that is true, you don't have to be in FESCo to create the policy or any proposal. 16:31
adamw just #action me to do a fesco ticket and i'll figure it out 16:31
Oxf13 poelcat: in fact, a subject matter expert may be a better choice to create such a policy, rather than the members of FESCo 16:32
* adamw is talking to security team now 16:32
jlaska #action adamw will reach out to FESCO for guidance on defining a security policy 16:32
poelcat Oxf13: yep, that is part of "make sure a security policy happens" 16:33
jlaska adamw: okay ... anything else I missed or need to capture? 16:33
Oxf13 poelcat: that also assumes that FESCo or the Fedora Board has determined that a security policy is even necessary. 16:34
poelcat we're off topic, let's move on :) 16:34
adamw jlaska: i think that's okay. 16:34
jlaska adamw: alright, thank you 16:34
jlaska #topic F-12 QA retrospective 16:35
jlaska #link https://www.redhat.com/archives/fedora-test-list/2009-November/msg01126.html 16:35
jlaska lots of good feedback on the wiki and mailing list already, so thanks to all who contributed so far 16:35
jlaska I'm still migrating mailing list feedback into the wiki page, I expect to wrap that up today 16:35
jlaska #link https://fedoraproject.org/wiki/Fedora_12_QA_Retrospective 16:36
jlaska As for next steps ... I plan to organize the feedback into related groups to make it easier to see trends/problem_areas 16:37
jlaska I'd like to then see some discussion around what the group wants to focus on for F-13 testing 16:37
jlaska I'm still uncertain on the most effective forum for that ... I'm guessing a mix of mailing list and a FUDCon break-out 16:38
jlaska this shouldn't be anything new ... we informally did this for F-12 16:38
jlaska all I'm trying to do now is make this process a bit more repeatable+transparent ... and hopefully help interested participants take part in our objectives for F-13 16:39
jlaska #info Next steps ... I plan to organize the feedback into related groups to make it easier to see trends/problem_areas 16:39
jlaska anyone else have questions or concerns on this topic? 16:40
adamw nothing really 16:41
Oxf13 nope 16:41
* Viking-Ice nothing from me.. 16:41
jlaska alrighty ... let's move on then 16:41
jlaska #topic AutoQA Updates 16:41
jlaska Just our usual check-in to see how things are progressing on the wwoods and kparal autoqa front 16:42
jlaska who wants to go first? 16:42
kparal ok, a few words from me :) 16:42
wwoods er, I'll go real quick 16:42
kparal heh 16:42
wwoods or.. yes 16:42
wwoods kparal: you go first 16:42
kparal alright 16:43
jlaska hehe, sorry guys ... I'll pick one of you first to avoid confusion next time 16:43
kparal there are a few patches in the autoqa mailing from me, maybe you have seen them 16:43
kparal the purpose is to make test development even easier 16:43
wwoods kparal: ooh, I didn't see the optparse patch for the watchers 16:44
kparal so all the watchers should have now (after accepting the patches) --help and --dry-run standardized 16:44
* wwoods was out of the office and off the VPN Nov. 25 until now, still catching up 16:44
kparal so now it could improve the expectations that people have when trying out the code 16:44
kparal I will start documenting the 'Getting started' stuff after it's in master 16:45
jlaska #info kparal has a few autoqa patches out for review - adding --help and --dry-run options to all watchers 16:45
kparal and I also worked a little bit on integrating rpmguard into autoqa, but not finished yet 16:45
jlaska #info kparal plans to contribute to the 'Getting started' use case once changes are in 16:46
* jlaska going crazy with meetbot tags ... sorry for annoyance 16:46
kparal wwoods, you may go on now :) 16:46
wwoods heh - thanks, kparal 16:47
jlaska #info kparal some progress on integrating rpmguard, more work remains 16:47
wwoods So last week I adapted kparal's patch for the autoqa harness to add the --local flag 16:47
wwoods so tests can be launched on your local system, to help with test development 16:47
wwoods it also supports the --dry-run flag, like the watchers 16:48
kparal already tried that, works perfect 16:48
wwoods kparal: great! 16:48
wwoods let's see - there were some fixes to the watchers 16:49
jlaska #info Wwoods accepted kparal's local autoqa harness patch ... tests can now be launched on your local system to facilitate test development/integration 16:49
wwoods they should be running autoqa only once for each repo/build that's updated, regardless of which (or how many) arches the build/tree/repo is available for 16:49
wwoods this is going to be important when we start working with more 'noarch' tests - tests like rpmlint, which don't need to run on the same arch as the target package/tree 16:50
wwoods also watch-repos.py was running tests too often, that was fixed 16:51
wwoods and I updated repoinfo to reflect the fact that there's no ppc rawhide anymore 16:51
* jlaska makes a note to ensure we're tracking the documentation needed for autotest system tagging 16:51
wwoods with the new repoinfo library that's just a simple config file change, so that's nice 16:51
wwoods in preparation for FUDCon I'm planning to write up some notes on AutoQA for Fedora developers 16:52
* jlaska was impressed with the repoinfo config file 16:52
wwoods and hopefully I'll be helping kparal get rpmguard running and sending emails and stuff 16:52
wwoods that's all from me. 16:53
jlaska wwoods: great stuff 16:53
* jlaska info's ... 16:53
jlaska #info wwoods improved support for running 'noarch' tests - tests like rpmlint, which don't need to run on the same arch as the target package/tree 16:53
jlaska #info wwoods updated repoinfo to reflect the fact that there's no ppc rawhide anymore 16:54
jlaska #info wwoods plans ... some FUDCon prep work and helping kparal integrate rpmguard 16:54
jlaska I think that gets it ... shout if I've missed anything 16:54
wwoods to be fair, the current support for 'noarch' tests is kind of a gross hack 16:54
jlaska we like hacks 16:55
wwoods but I laid some of the groundwork to make it work sanely later 16:55
* jlaska makes a note to head to cafepress after meeting :) 16:55
skvidal wwoods: repoinfo is in autoqa? 16:55
wwoods skvidal: yes 16:55
skvidal wwoods: thanks 16:55
jlaska Just a few updates on some of my autoqa action items ... 16:56
jlaska #info thanks to help from abadger1999, autoqa-israwhidebroken is now packaged. I sent a link to the branch to autoqa-devel. If folks are happy, I can merge that into master 16:56
jlaska having this packaged highlighted a few next steps for having this service run as http://admin.fedoraproject.org/israwhidebroken 16:57
jlaska I don't think I'll be able to knock those out before FUDCon, so might look for guidance from abadger1999 and/or mmcgrath if they have the time 16:57
jlaska wwoods: kparal: anything else on the autoqa front? 16:58
kparal not from me 16:58
jlaska kparal: okay ... oh, wwoods said he was good earlier ... alright, next up ... 16:59
jlaska #topic Open Discussion - <your topic here> 16:59
adamw FINALLY time for the week's most important topic 16:59
jlaska any topics not yet mentioned people would like to discuss? 16:59
jlaska adamw: hit it! 16:59
adamw i move that glogg be adopted as the new official drink of QA 16:59
adamw http://loupgaroublond.blogspot.com/2009/11/glogg.html 16:59
adamw it involves wine, port and whiskey, and has an awesome name. i see no drawbacks. 16:59
jlaska #topic Open Discussion - Glögg as the official drink of QA 17:00
wwoods glogg!! 17:00
wwoods I have a bottle of that in my house right now 17:00
jlaska adamw: if we drink too much, side affects? 17:00
Oxf13 adamw: I was thinking more 'Magic Hobo Gravy' http://hijinksensue.com/2009/11/27/the-special-sauce/ 17:00
jlaska "Best code I ever wrote!" 17:00
kparal I want pictures! :) 17:00
adamw i do not understand what you mean by 'too much' 17:00
adamw this seems a nonsensical concept 17:00
jlaska the first images.google.com result for grogg is not what I'd expect 17:01
jlaska adamw: :) 17:01
jlaska http://www.mostphotos.com/preview/150009/grogg-drink-ice-glas-red.jpg ? 17:01
adamw iced grogg? 17:01
jlaska http://www.drunkenblog.com/drunkenblog-archives/i/oh_glogg_youdevil.jpg 17:01
Oxf13 jlaska: "grogg" vs "glogg" 17:01
wwoods yeah I think maybe Glögg and grog(g) are different things 17:01
adamw http://www.acatinthekitchen.com/photo/advent/glogg.jpg 17:01
jlaska indeed they are ... never trust "Maybe you meant grogg" :) 17:02
jlaska kparal: I've seen this a lot http://www.saturnus.se/images/produkter/glogg_export.jpg 17:02
jlaska okay ... anything else on the radar we need to discuss? 17:02
kparal never seen that in czech republic 17:02
Oxf13 yeah, who is going to be on the bus? 17:03
jlaska kparal: You've got some of the best stuff on earth right in your back yard :) 17:03
jlaska #topic Open Discussion - FUDCon travel plans 17:03
jlaska #topic Open Discussion - FUDCon travel plans 17:04
jlaska nirik: thanks! 17:04
wwoods jlaska and I will be flying into Toronto on Friday. Remember yer passports, USians. 17:04
* jlaska wonders when the next EMEA FUDCon is 17:05
jlaska okay ... I'll close out the meeting in 2 minutes unless any urgent items come up 17:06
jlaska alright folks ... I think we can close it out 17:08
jlaska thanks for your time 17:08
jlaska As always, minutes will be sent to the list 17:08
jlaska #endmeeting 17:08

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!

Retrieved from "https://fedoraproject.org/w/index.php?title=QA/Meetings/20091130&oldid=139473"