m (internal link cleaning) |
(same thing with another link) |
||
(One intermediate revision by the same user not shown) | |||
Line 62: | Line 62: | ||
Jlaska noted that Liam and Hurry are doing a lot of heavy lifting with regards to beefing up the installation test plan. There are several wiki improvements coming aimed to improve posting test results, new test cases for yum repos, a lot of improvements to existing tests and a reorganization of priorities based on the updated release criteria. | Jlaska noted that Liam and Hurry are doing a lot of heavy lifting with regards to beefing up the installation test plan. There are several wiki improvements coming aimed to improve posting test results, new test cases for yum repos, a lot of improvements to existing tests and a reorganization of priorities based on the updated release criteria. | ||
Adam reminded the group that a ticket has been filed for FESCO review regarding a security checklist/test plan. Adam also started drafting test cases to cover the desktop verification from [[Fedora_Release_Criteria]] (see | Adam reminded the group that a ticket has been filed for FESCO review regarding a security checklist/test plan. Adam also started drafting test cases to cover the desktop verification from [[Fedora_Release_Criteria]] (see https://fedoraproject.org/wiki/Category:Desktop_Acceptance_Test_Cases). | ||
Viking-Ice raised the topic of creating an xorg-x11-drv test plan. Adam mentioned that this was a big discussion topic during the team sprint. The problem space is that we want to test available X hardware. The community provides this pool of hardware, but we have no means to gather and organize feedback in the short timeframe that would be meaningful for the X developers. The current test coverage for X consists of driver-specific test days (intel, nouveau and radeon), and providing information to aid testers to properly triage failures (see [[How_to_debug_Xorg_problems]]). | Viking-Ice raised the topic of creating an xorg-x11-drv test plan. Adam mentioned that this was a big discussion topic during the team sprint. The problem space is that we want to test available X hardware. The community provides this pool of hardware, but we have no means to gather and organize feedback in the short timeframe that would be meaningful for the X developers. The current test coverage for X consists of driver-specific test days (intel, nouveau and radeon), and providing information to aid testers to properly triage failures (see [[How_to_debug_Xorg_problems]]). | ||
Line 559: | Line 559: | ||
|- id="t16:32:26" | |- id="t16:32:26" | ||
! style="background-color: #4b904b" | kparal | ! style="background-color: #4b904b" | kparal | ||
| style="color: #4b904b" | I noticed adamw is creating test plan for the desktop stuff: | | style="color: #4b904b" | I noticed adamw is creating test plan for the desktop stuff: https://fedoraproject.org/wiki/Category:Desktop_Acceptance_Test_Cases | ||
|| [[#t16:32:26|16:32]] | || [[#t16:32:26|16:32]] | ||
|- id="t16:32:37" | |- id="t16:32:37" |
Latest revision as of 16:55, 12 May 2022
Attendees
People present (lines said)
- jlaska (154)
- adamw (57)
- Viking-Ice (19)
- maxamillion (17)
- wwoods (12)
- kparal (8)
- nirik (4)
- zodbot (3)
- buggbot (1)
Regrets:
Agenda
- Proposed meeting agenda
- [FIXME meetbot summary]
Previous meeting follow-up
- jlaska to post recommendations on F-12 QA retrospective
- adamw to escalate security policy issue to fesco
- jlaska to set up a meeting to co-ordinate test plan changes for F13
- Hosted a small team sprint the week prior to Christmas with Liam, Hurry, Adam and myself to work through open F-13 test planning tickets
- adamw and jlaska to look at setting up f13 test day process
- Ideas posted to Talk:QA/Fedora_13_test_days
- Adam and James to continue discussion
F-13 Recommendations
Discussion around the proposed QA changes for F-13 based on Fedora_12_QA_Retrospective#Recommendations. Begin identifying gaps and owners.
Process
See Fedora_12_QA_Retrospective#Process_4
Move User:Liam/Draft_Install_Test_SOP out of draft status - to me this is part of making install testing easier.
Move User:Poelstra/blocker_bug_meeting_sop out of draft status - adamw had some improvements during FUDCon, I think we just need to dust it off before the blocker reviews kick in again
Draft a SOP (or extract from existing BugsAndFeatureRequests page) that describes how to document bugs (e.g. common_bugs or release notes or install guide etc...). Adamw noted that while no single page exists now, each location includes information on how to contribute. Adamw and Jlaska wondered if there was a way to provide a single page that links to existing documentation so that we can easily direct the efforts of testers who wish to provide documentation/workarounds for bugs they've found. jlaska took action to reach out to beland for guidance/ideas on how to document the process (or point to existing documentation) for how bugs are noted (common_bugs, release notes, install guide etc...) How to determine which bugs land in which place
Draft/promote a recommended test checklist for rawhide testers (for example see User:Jlaska/Draft2) - this is something spot discussed a while ago and adamw drafted an idea. I think it would be worthwhile to reference from the QA/Joining page ... ideas?
'How to debug <component> problems?' -- Determine whether guides can be offered by Fedora QA as a result of a test day/feature process. Viking-Ice asked for more details. Jlaska explained that this was largely intended to follow the successful model Viking-Ice initiated by starting the debugging guide for the F-12 dracut feature (see How_to_debug_Dracut_problems). Viking-Ice offered to review the list of upcoming features for good debug guide candidates (see Releases/13/FeatureList).
Adamw offered assistance adding/improving installation testing to the current QA activities page.
Planning
See Fedora_12_QA_Retrospective#Test_Planning_3
Jlaska noted that Liam and Hurry are doing a lot of heavy lifting with regards to beefing up the installation test plan. There are several wiki improvements coming aimed to improve posting test results, new test cases for yum repos, a lot of improvements to existing tests and a reorganization of priorities based on the updated release criteria.
Adam reminded the group that a ticket has been filed for FESCO review regarding a security checklist/test plan. Adam also started drafting test cases to cover the desktop verification from Fedora_Release_Criteria (see https://fedoraproject.org/wiki/Category:Desktop_Acceptance_Test_Cases).
Viking-Ice raised the topic of creating an xorg-x11-drv test plan. Adam mentioned that this was a big discussion topic during the team sprint. The problem space is that we want to test available X hardware. The community provides this pool of hardware, but we have no means to gather and organize feedback in the short timeframe that would be meaningful for the X developers. The current test coverage for X consists of driver-specific test days (intel, nouveau and radeon), and providing information to aid testers to properly triage failures (see How_to_debug_Xorg_problems).
Viking-Ice, nirik and Adamw discussed fedoraforum.org user feedback. Adam felt that the 2 biggest areas of feedback were around 1) HELP MY X/COMPIZ DOESN'T WORK and 2) HELP MY PROPRIETARY DRIVER DOESN'T WORK. The first area is already covered, and the second point is out of our scope.
Viking-Ice took action to investigate adding an LXDE test day to the F-13 test day schedule. Similarly, maxamillion took action to schedule an XFCE test day on the F-13 test day schedule.
Jlaska took an action item to reach out to User:Mclasen for information on any Fit'n'Finish test day plans for F-13.
Communication
See Fedora_12_QA_Retrospective#Communication_4
Adam offered assistance improving the QA wiki pages to include any install activities. Jlaska suggested that Hurry had some initial ideas here, and perhaps they could sync up.
Automation
See Fedora_12_QA_Retrospective#Test_Automation
Not a lot of discussion on this point during hte meeting since each meeting typically devotes time AutoQA status updates.
While discussing 'Deploying AutoQA 'is rawhide broken?', Jlaska was reminded to reach out to fedora-{test,devel}-list for trading package reviews.
Some discussion followed on whether rpmlint and rpmguard should be merged. Kparal suggested that the two projects have different goals and there is value in keeping them separate.
Wwoods noted that assisting Liam and Hurry by helping to build the autoqa install automation fundamentals would be a focus area for the new year. Will mentioned the inputs needed for any install test; 1) a description of the test system, 2) a URL to a kickstart file, 3) a URL to an install source. Jlaska noted that he included those three points in an email to Liam before the break. Jlaska will catch up with Liam to see if he is having fun with python.
Open discussion - <Your topic here>
Upcoming QA events
- 2010-01-21 - Pre-Alpha Rawhide Acceptance Test Plan #1
- 2010-01-28 - Pre-Alpha Rawhide Acceptance Test Plan #2
- 2010-01-04 - Pre-Alpha Rawhide Acceptance Test Plan #3
- 2010-02-05 - Alpha Blocker Meeting (F13Alpha) #1
- 2010-02-11 - Test Alpha 'Test Compose' (boot media testing)
- 2010-02-12 - Alpha Blocker Meeting (F13Alpha) #2
Action items
- Viking-Ice investigating adding an LXDE test day to the F-13 test day schedule
- maxamillion investigating scheduling a XFCE test day to F-13 test day schedule
- adamw and rhe to discuss ways to add install testing as a QA activity
- jlaska will reach out to mclasen for thoughts on F-13 fit'n'finish test days
- jlaska to reach out ot fedora-{test,devel}-list for trading reviews
- jlaska to reach out to beland for guidance/ideas on how to document the process (or point to existing documentation) for how bugs are noted (common_bugs, release notes, install guide etc...) How to determine which bugs land in which place
IRC transcript
jlaska | #startmeeting Fedora QA Meeting | 16:01 |
---|---|---|
zodbot | Meeting started Mon Jan 4 16:01:14 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
zodbot | Useful Commands: #action #agreed #halp #info #idea #link #topic. | 16:01 |
jlaska | #meetingname qa | 16:01 |
zodbot | The meeting name has been set to 'qa' | 16:01 |
jlaska | #topic gathering | 16:01 |
adamw | hellllo | 16:02 |
jlaska | Howdy folks ... let's do a show of hands | 16:02 |
* wwoods shows off his hands | 16:02 | |
jlaska | ooh, look at those hands | 16:02 |
wwoods | theyr'e the new '10 model | 16:02 |
* kparal waves too | 16:02 | |
adamw | who'd you cut them off? and where'd you get the jars? | 16:02 |
* Viking-Ice raises his hand for a high five on the new year.. | 16:02 | |
wwoods | Viking-Ice: o/ \o | 16:03 |
jlaska | down low | 16:03 |
adamw | wwoods can give you a high twenty | 16:03 |
jlaska | alright ... let's get things started | 16:04 |
jlaska | we've got a light agenda today ... but always room for 'open discussion' | 16:04 |
jlaska | #topic Previous meeting follow-up | 16:04 |
jlaska | okay ... trying to figure out which meeting was the previous | 16:04 |
jlaska | the last set of tasks I'm tracking are listed at QA/Meetings/20100104#Previous_meeting_follow-up | 16:05 |
jlaska | #info jlaska to post recommendations on F-12 QA retrospective | 16:05 |
jlaska | these are posted and available at Fedora_12_QA_Retrospective#Recommendations | 16:05 |
jlaska | I've left most of the agenda for walking through this | 16:05 |
jlaska | next I've got ... | 16:06 |
jlaska | #info adamw to escalate security policy issue to fesco | 16:06 |
jlaska | I don't have any updates on that one ... did that already happen? | 16:07 |
adamw | err. i think i filed a ticket. just a sec | 16:07 |
adamw | yeah, I did | 16:07 |
adamw | https://fedorahosted.org/fesco/ticket/297 | 16:07 |
jlaska | ah great | 16:08 |
adamw | no reply yet | 16:08 |
* jlaska lost the linky | 16:08 | |
jlaska | so that'll come up for discussion at the next FESCO meeting right? | 16:08 |
jlaska | Next up ... | 16:08 |
jlaska | #info jlaska to set up a meeting to co-ordinate test plan changes for F13 | 16:08 |
jlaska | We had a quick sprint the week prior to Christmas with adamw, lili, rhe and myself walking through the list of F-13 test plan tickets | 16:09 |
jlaska | still lots of kinks to iron out in the sprint process, and there are still a few inprogress tickets | 16:10 |
* jlaska searches for ticket query | 16:10 | |
jlaska | https://fedorahosted.org/fedora-qa/query?status=new&status=assigned&status=reopened&component=Test+Review&milestone=Fedora+13&order=priority | 16:11 |
jlaska | I'll circle back and bump the status on a few of those later today | 16:11 |
jlaska | last item ... | 16:11 |
jlaska | #info adamw and jlaska to look at setting up f13 test day process | 16:11 |
jlaska | No updates from me on this ... I'd like to chat w/ adamw this week to iron out an approach | 16:12 |
maxamillion | is there something that is going to be greatly different from the last test day process(es)? | 16:12 |
jlaska | we've got initial ideas posted to the wiki already ... Talk:QA/Fedora_13_test_days | 16:12 |
jlaska | maxamillion: I don't expect so ... just need to get on top of it and actively work the schedule | 16:13 |
jlaska | okay, anything not mentioned? | 16:13 |
maxamillion | ah, nice | 16:13 |
maxamillion | I'll be sure to read that later | 16:13 |
jlaska | if not ... we'll jump into the F-13 QA recommendations | 16:14 |
jlaska | maxamillion: cool, and of course, test day topic suggestions encouraged too! | 16:14 |
jlaska | #topic F-13 QA Recommendations | 16:14 |
jlaska | So ... I've taking all the QA retrospective feedback and tried to organize things into several groups | 16:14 |
maxamillion | jlaska: I'll be working with kevin and christoph on the xfce spin test day plans and try to add suggestions if I find any :) | 16:15 |
jlaska | maxamillion: awesome, that's just what I had in mind. Any additional spin test days would be fun (much like the sugar event) | 16:15 |
Viking-Ice | I was thinking about going knee deep in LXDE so I should be able to write something.. | 16:15 |
maxamillion | Viking-Ice: be sure to touch base with cwickert, i'm sure he'd welcome the interest in LXDE | 16:16 |
jlaska | so I've organized the feedback into the following themes: process, planning, communication and automation | 16:16 |
jlaska | these aren't hard themes, just what bubbled up from how I read the data | 16:16 |
jlaska | so there is overlap between these themes .. but in general, I was happy with the grouping | 16:17 |
Viking-Ice | maxamillion: I was planning to hear from him.. | 16:17 |
maxamillion | Viking-Ice: cool cool :) | 16:17 |
jlaska | maxamillion: Viking-Ice should I add an action item for next week on this? or too soon? | 16:17 |
jlaska | ^--> re: LXDE and XFCE spin test days? | 16:17 |
Viking-Ice | you can add action item for me | 16:18 |
jlaska | okay | 16:18 |
maxamillion | we could do next week, that would give me a bit of motivation to get on the ball about it | 16:18 |
maxamillion | errr I* | 16:18 |
* Viking-Ice works best under pressure.. | 16:18 | |
jlaska | #action Viking-Ice investigating adding an LXDE test day to the F-13 test day schedule | 16:18 |
jlaska | #action maxamillion investigating scheduling a XFCE test day to F-13 test day schedule | 16:19 |
maxamillion | w00t! my first action item in the QA world :P | 16:20 |
jlaska | maxamillion: heh, you've arrived :D | 16:20 |
jlaska | so have folks had a chance to look through the recommendations? | 16:20 |
jlaska | Fedora_12_QA_Retrospective#Recommendations | 16:20 |
jlaska | there shouldn't be surprises there ... I've basically reorganized the feedback provided | 16:21 |
jlaska | what I'd like to do is walk through the list and see if we can identify owners for some of the tasks | 16:21 |
maxamillion | jlaska: do you want to #link that so its in the minutes? (that might not even be the right tag, but you get the idea) :) | 16:21 |
jlaska | maxamillion: sure ... | 16:21 |
jlaska | #link Fedora_12_QA_Retrospective#Recommendations | 16:22 |
adamw | just refreshing my memory | 16:22 |
jlaska | many of the listed recommendations are things several of you are already on top of | 16:23 |
adamw | i could take "Promote install testing as a QA activity" | 16:23 |
jlaska | adamw: thanks, Hurry has some ideas on this already ... would you two like to sync up? | 16:24 |
jlaska | perhaps her experience on the install side iwth your wiki mastery can come up with some good ideas | 16:24 |
adamw | also most of the 'communication' stuff | 16:24 |
adamw | well yeah i was just meaning i could do the wiki bits | 16:25 |
adamw | stick me in a #action | 16:25 |
Viking-Ice | I need a little explanation with this "Determine whether guides can be offered by Fedora QA as a result of a test day/feature process" | 16:25 |
jlaska | #action adamw and rhe to discuss ways to add install testing as a QA activity | 16:25 |
jlaska | adamw: kparal had some social networking suggestions already, perhaps we can divide and conquer there? | 16:26 |
jlaska | Viking-Ice: my thinking there was to follow your lead for dracut | 16:26 |
kparal | not really mine suggestions, but I can help | 16:26 |
jlaska | kparal: oh right ...these were from the lunch'n'learn | 16:26 |
kparal | correct | 16:27 |
Viking-Ice | Advertice on the Fedora Project network on FB | 16:27 |
jlaska | Viking-Ice: basically, that dracut debug page rocks ... I'd like to see we can follow that model again for maybe a few features? | 16:27 |
jlaska | what do you think? | 16:27 |
Viking-Ice | sounds raz.. | 16:27 |
Viking-Ice | I'll run through the feature stuff and see what I can write.. | 16:28 |
jlaska | Viking-Ice: sweet, I think that would be tremendously helpful. I just don't know a good way to target specfiic features | 16:28 |
jlaska | dracut was a good choice since it was something *everyone* would need | 16:28 |
jlaska | perhaps python3 might be similar in that regard? or others | 16:29 |
jlaska | #link Releases/13/FeatureList | 16:30 |
jlaska | #topic F-13 QA Recommendations - test planning | 16:30 |
jlaska | on the test planning front ... much of this is already underway | 16:31 |
jlaska | #info Maintenance of the Fedora Install Test Plan - Liam and Hurry are doing a lot of heavy lifting here | 16:31 |
jlaska | #info Draft a more formal, but basic and lightweight, security checklist/test plan - discussed earlier, adamw has a fesco ticket filed ... stay tuned | 16:31 |
jlaska | #info Draft a test plan to cover the desktop verification from Fedora_Release_Criteria | 16:32 |
jlaska | #link Fedora_Release_Criteria | 16:32 |
kparal | I noticed adamw is creating test plan for the desktop stuff: https://fedoraproject.org/wiki/Category:Desktop_Acceptance_Test_Cases | 16:32 |
adamw | yeah, i'm in teh middle of that | 16:32 |
adamw | got about halfway through before the xmas break | 16:32 |
adamw | will be back on it this week | 16:32 |
jlaska | kparal: perfect, thanks for the link | 16:32 |
kparal | I will gladly help with that if time permits | 16:33 |
adamw | sure, if anyone wants to chip in tests that'd be great | 16:33 |
Viking-Ice | I think we need to get in touch with the X team to come up with an formal Xorg test plan ( not just x11-drv-* ) | 16:33 |
jlaska | Viking-Ice: good segway ... that was next | 16:33 |
adamw | well we talked about that quite a lot at the sprint | 16:33 |
jlaska | #info We have no formal Xorg-x11-drv test plan ... either create one, or develop proposal for gathering the same feedback through other means (smolt, bugzilla) | 16:33 |
adamw | i actually don't think a test plan for X would achieve much | 16:33 |
adamw | we started doing a test matrix then realized it was pretty pointless | 16:33 |
jlaska | adamw: can you talk more about how we came to that conclusion? | 16:34 |
adamw | the tricky thing with X is that the main problem with it is that we're dealing with seventy zillion different bits of hardware | 16:34 |
jlaska | yeah that ---^ :) | 16:34 |
adamw | and the level of functionality we currently care about is pretty basic, this isn't really deep level stuff | 16:34 |
maxamillion | I think there could be a matrix of hardware and "Works for Me" and maybe some break down of how much works (KMS, direct rendering, etc) | 16:34 |
adamw | the types of problems we get with X are not 'this one out of the millions of really complex levels of functionality suddenly regressed' but 'o shit this card doesn't work any more' | 16:35 |
adamw | maxamillion: that would rapidly get HORRIBLY unsustainable | 16:35 |
jlaska | maxamillion: if someone wanted to look into how to pull this out of smolt and present the information ... that would be a fun project | 16:35 |
adamw | i really can't emphasize the amount of hardware we're talking about here enough | 16:35 |
maxamillion | adamw: for a test day? | 16:35 |
Viking-Ice | what about input ( keyboards usb etc ) and for example multitouch ? | 16:35 |
maxamillion | or you mean long term | 16:35 |
adamw | we already do test days | 16:35 |
maxamillion | ? | 16:35 |
maxamillion | oh ok, i misunderstood | 16:35 |
adamw | we're talking about some kind of test plan | 16:35 |
maxamillion | sorry, I'm in a meeting at $dayjob and trying to keep up with this one ... don't mind me :P | 16:36 |
adamw | i just don't think that's an approach that fixes the X situation; it's not the paradigm that addresses the kinds of problems we actually get with X | 16:36 |
maxamillion | adamw: yeah, I agree | 16:36 |
jlaska | maxamillion: no it's a good suggestion ... it's just trying to figure out how to sustainably execute the plan and track the data | 16:36 |
adamw | Viking-Ice: um. in theory an input matrix is a useful idea, but in practice it kinda depends how much we care about anything besides basic mouse+keyboard stuff | 16:36 |
adamw | er, s/matrix/test plan/ | 16:37 |
adamw | multitouch is way too basic at this point to worry about | 16:37 |
adamw | you can't really _use_ it in any meaningful way yet | 16:37 |
adamw | jlaska still reckons some kind of test plan for X is a good idea, but i still don't quite understand exactly what we can achieve with one that we're not achieving already; this is where we left it in the sprint :) | 16:37 |
jlaska | one way we can use the existing test cases is much how they've been used so far | 16:37 |
jlaska | for test days and for use to help isolate the true failure of reported bugs | 16:38 |
Viking-Ice | adamw: Ok Talking about meaningful did we receive any feed back from nirik and others on the front ( forums #fedora etc ) about what they get bombarded with ( perhaps something we should track and target for test days ) | 16:39 |
jlaska | Viking-Ice: good suggestion | 16:39 |
* nirik looks up. | 16:39 | |
adamw | well, i'm mostly on teh front for X stuff, and yeah, we do co-ordinate a bit on that | 16:39 |
nirik | you mean specific stuff? or in general areas we see a lot of stuff? | 16:40 |
jlaska | nirik: other soft spots from the F-12 release that could modify the QA plan for F-13 | 16:40 |
Viking-Ice | all proprietary aside the issues most end users are faced with | 16:40 |
adamw | i don't think we can add much to our process based on that feedback. i'd say broadly what you get splits into two camps: 'HELP MY X/COMPIZ DOESN'T WORK' and 'HELP MY PROPRIETARY DRIVER DOESN'T WORK' | 16:40 |
adamw | second is out of our scope | 16:40 |
adamw | first is already covered | 16:40 |
* nirik nods. | 16:40 | |
jlaska | true | 16:40 |
jlaska | okay ... last I have on the planning side is Test Days | 16:41 |
adamw | as i said, X is fundamentally a shallow area: the issues are due to hardware variety (and the developers rewriting everything every three months). there just isn't a complex tangle of functionality to test | 16:41 |
jlaska | adamw: if we had a pull of every hardware adapter available ... would there be some type of feedback to provide to X maintainers that would be valuable? | 16:42 |
nirik | having the list of workarounds especially for install has been great on the common bugs list. | 16:42 |
jlaska | oh god ... s/pull/pool/ | 16:42 |
adamw | well, yeah. if we had a Big Test Lab with every graphics adapter in the world in it, sure. what they'd be interested in is knowing with every day's rawhide which systems basically worked. for bonus points, which could do enough 3D to run compiz. | 16:43 |
jlaska | heh | 16:43 |
adamw | i mean, that's really it. like i keep saying, it's not complicated =) | 16:43 |
jlaska | adamw: so instead, we have a highly distributed test lab | 16:43 |
adamw | there was that thread about 'stop working on rawhide and work on f12' or whatever on devel-list | 16:43 |
wwoods | (if you consider the set of Interested Fedora Users as a distributed test lab..) | 16:43 |
adamw | dave airlie (or ajax, I forget) posted the X team's List Of Things We Really Care About to that thread, which is basically what I just said | 16:44 |
jlaska | wwoods: true true | 16:44 |
jlaska | adamw: gotcha | 16:44 |
jlaska | alright ... so last I had on the planning topic was test days | 16:44 |
jlaska | anyone know if they are plans to revisit the 'fit and finish' events? | 16:44 |
Viking-Ice | adamw: Big Test Lab the world not enough ;) | 16:44 |
jlaska | if not, I can follow-up w/ mclasen | 16:44 |
adamw | sure. how i look at that is that the blocking points in that way of looking at things all involve making the lab work efficiently. the test plan is not part of that. the test plan defines _what to test_, which as I keep saying is really simple and everyone knows it already | 16:44 |
adamw | all the problems with X testing, if we look at the 'the world is a big test lab' theory, lie in actually co-ordinating the distributed systems and people who make up the 'lab' to do testing and provide useful info. which a test plan can't really help with, i don' think. but hey. sorry, jlaska wants to move on :) | 16:45 |
jlaska | adamw, no thanks ... you explained it better than I could have hoped to | 16:46 |
Viking-Ice | Let's move along.. | 16:46 |
jlaska | #action jlaska will reach out to mclasen for thoughts on F-13 fit'n'finish test days | 16:47 |
jlaska | okay ... any other suggestions on the test planning side for F-13? | 16:47 |
jlaska | for reference ... Fedora_12_QA_Retrospective#Test_Planning_3 | 16:47 |
jlaska | alright ... the next group is where wwoods and kparal rock | 16:48 |
jlaska | #topic F-13 QA Recommendations - test automation | 16:48 |
jlaska | #link Fedora_12_QA_Retrospective#Test_Automation | 16:48 |
jlaska | The first obvious one is ... | 16:48 |
jlaska | #info Deploy AutoQA 'is rawhide broken?' front-end into fedora infrastructure | 16:49 |
jlaska | this is inprogress ... and going through package review at the moment | 16:49 |
jlaska | #action jlaska to reach out ot fedora-{test,devel}-list for trading reviews | 16:49 |
jlaska | #info Deploy build-time sanity checking mechanism for package maintainers (rpmguard, rpmlint etc...) | 16:49 |
jlaska | Kamil and Will are knee deep in this now | 16:49 |
jlaska | with both rpmlint and rpmguard working into the system | 16:50 |
kparal | and will be deeper :) | 16:50 |
jlaska | hehe | 16:50 |
jlaska | The last item I have here is | 16:50 |
jlaska | #ifo Develop proof-of-concept for automating install test matrix | 16:50 |
jlaska | #info Develop proof-of-concept for automating install test matrix | 16:50 |
wwoods | I'm pretty sure rpmguard will obsolete rpmlint eventually | 16:50 |
kparal | I don't think so, they have different goal | 16:51 |
jlaska | Liam and Hurry are working towards a automated DVD install test at the moment ... and then likely will need wwoods expertise for integrating the tests into AutoQA | 16:51 |
wwoods | kparal: hm, good point | 16:51 |
wwoods | jlaska: yeah, I think this will probably be my first big goal for the new year? | 16:51 |
jlaska | wwoods: cool, this would be huge to see traction on before F-13 kicks into high gear | 16:52 |
jlaska | Liam was looking into playing with install.py over the break and expanding it to do a basic DVD install ... I'll be checking in to see how he's making out soon | 16:52 |
jlaska | anything else under automation that wasn't covered ... that needs to be on the radar? | 16:52 |
wwoods | yeah, IIRC it involves designing a test that takes as input: | 16:52 |
wwoods | 1) a description of the test system hardware (disks, disk sizes), | 16:53 |
wwoods | 2) a kickstart (which describes partitioning etc), | 16:53 |
wwoods | 3) an installable tree/image | 16:53 |
jlaska | yeah, I should have cc'd you on the mail I sent him | 16:53 |
jlaska | basically the same list of inputs | 16:53 |
wwoods | pretty sure a test that would do that would cover the entire test matrix, but we'll see | 16:53 |
jlaska | okay ... we touched on each of the other sections already but here they are for the logs | 16:54 |
jlaska | #topic F-13 QA Recommendations - process | 16:55 |
jlaska | #link Fedora_12_QA_Retrospective#Process_4 | 16:55 |
Viking-Ice | One thing that came to my mind when I hit bug 549944 is this something we can automate an test for ( looks like version mismatch ) | 16:55 |
buggbot | Bug https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=549944 medium, low, ---, jskala, NEW, Updating ppp to 2.4.5-x breaks NetworkManager-pptp vpn connection... | 16:55 |
jlaska | Viking-Ice: hrmm, yeah if there's something there that could have been detecting with rpmguard, we can get a ticket in for a future test case idea | 16:56 |
jlaska | so for process ... I had a few things | 16:57 |
jlaska | #info Move User:Liam/Draft_Install_Test_SOP out of draft status - to me this is part of making install testing easier | 16:57 |
jlaska | #info Move User:Poelstra/blocker_bug_meeting_sop out of draft status - adamw had some improvements during FUDCon, I think we just need to dust it off before the blocker reviews kick in again | 16:57 |
jlaska | #info Draft a SOP (or extract from existing BugsAndFeatureRequests page) that describes how to document bugs (e.g. common_bugs or release notes or install guide etc...) | 16:58 |
jlaska | ^^^ I'm a simple caveman ... so the docs might already be out there, but I could stand to be reminded how we advise folks on documenting issues | 16:58 |
adamw | we don't have anything really | 16:58 |
adamw | well, each document advises people how to contribute to it | 16:59 |
adamw | the common_bugs page advises how to add to the common_bugs page | 16:59 |
jlaska | #info Draft/promote a recommended test checklist for rawhide testers (for example see User:Jlaska/Draft2) - this is something spot discussed a while ago and adamw drafted an idea. I think it would be worthwhile to reference from the QA/Joining page ... ideas? | 16:59 |
adamw | the docs team have a page explaining how to contribute to the release notes | 16:59 |
adamw | jlaska: we could probably stick it up as a page and just link it from one of the existing sections on Joining | 16:59 |
adamw | off the top of my head | 16:59 |
jlaska | adamw: yeah true ... I was looking for a single page that had pointed to the instructions and helped advise which place was relevant for different bugs | 17:00 |
jlaska | #info # 'How to debug <component> problems?' -- Determine whether guides can be offered by Fedora QA as a result of a test day/feature process. - Viking-Ice already discussed this previously | 17:00 |
adamw | jlaska: the problem with that is, what would drive people to the central page? | 17:00 |
jlaska | just like all other debug guides and doc pages ... we would | 17:01 |
jlaska | in our IRC discussions, mailing list interactions etc | 17:01 |
adamw | jlaska: well i don't see a natural flow, a real role for this central page | 17:01 |
adamw | hmm, i see | 17:01 |
jlaska | you don't .. cause you are the natural flow for this stuff! :D | 17:01 |
adamw | well that's what i'm trying to explain in a crappy way =) | 17:01 |
jlaska | hehe | 17:01 |
jlaska | I can take that action on that one for the release | 17:02 |
adamw | usually i can see a sorta natural 'place' for a new wiki page, where it would fit into the flow of reference from page to page...i'm not sure i can see a real good place for something new here...maybe beland would have an idea how to structure it | 17:02 |
jlaska | oooh, yeah he has a way with those kinds of wiki changes | 17:02 |
jlaska | I can reach out to beland for guidance/ideas if that makes sense | 17:03 |
adamw | sure | 17:03 |
adamw | maybe he can come up with a mockup that'd make me go 'ohhh yeah' | 17:03 |
jlaska | okay | 17:03 |
jlaska | #action jlaska to reach out to beland for guidance/ideas on how to document the process (or point to existing documentation) for how bugs are noted (common_bugs, release notes, install guide etc...) How to determine which bugs land in which place | 17:04 |
jlaska | #topic F-13 QA Recommendations - communication | 17:05 |
jlaska | #link Fedora_12_QA_Retrospective#Communication_4 | 17:05 |
jlaska | I'm not going to spend time on this section, we discussed it earlier | 17:05 |
jlaska | so ... any other thoughts/concerns on the recommendations | 17:05 |
jlaska | I think we have a lot of good ideas | 17:05 |
jlaska | and I'd like for us to look back on this page after F-13 ... and say ... great, we did X% of what we said we would | 17:06 |
jlaska | regardless of what the value of X is | 17:06 |
jlaska | okay ... let's move on to open discussion | 17:06 |
jlaska | #topic Open discussion - <your topic here> | 17:06 |
adamw | nothing here | 17:08 |
jlaska | <crickets> | 17:08 |
jlaska | well, happy new year everyone and welcome back :) | 17:08 |
jlaska | I'm sure folks are still digging out from under email | 17:08 |
Viking-Ice | I propose that Fedora QA uses it's extensive forces and influences in the whole world to get a softer toilet paper to public toilets and conduct massive everyday testing on those paper.. | 17:08 |
Viking-Ice | in the year 2010 | 17:09 |
jlaska | Viking-Ice: I'd like to see that test plan! | 17:09 |
kparal | :)) | 17:09 |
Viking-Ice | I need to redraw it I accidentally flushed it down the drain.. | 17:09 |
jlaska | well, on that note ... | 17:10 |
jlaska | #endmeeting | 17:10 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!