From Fedora Project Wiki

< QA‎ | Meetings
Revision as of 20:34, 15 February 2010 by Jlaska (talk | contribs) (Add irc transcript)

Attendees

People present (lines said)

Regrets:

Agenda

Previous meeting follow-up

  1. jlaska will follow-up w/ nirik after the meeting to setup zodbot feed watchers
  2. jlaska to work w/ poelstra to give 'release validation' spin to quality milestones

Rawhide Acceptance Testing

Test Summary (see recap)

Next Steps:

  • Fix AutoQA post-tree-compose hook to monitor for RAT droppings (see ticket#115)
  • Commit reviewed patches and rework one proposed change.

Alpha TC test status

Alpha TC#2 is available for testing. Instructions for downloading and providing feedback available at:

IDEA - Live images for test compose milestone

Adamw proposed including live images with future 'test compose' milestones?

IDEA - Unique ISO image names for each drop

kparal proposed using unique ISO file names for each official test drop.

AutoQA project update

Deps/conflicts prevention

Last week

This week

  • Build a testsuite to validate depchecks functionality (possibly using rpmfluff)

Results collection

Last week

  • Held design session last week with wwoods, kparal and jskladan (see recap)

install automation

Last week

This week

  • Chinese new year, so no action expected.

packaging/deployment

Last week

  • Akurtakov's help has been tremendous ... more deps keep popping up and he has helped play wack-a-mole

This week

  • I still don't have a final picture for what needs packaging. I'd like to schedule some sit-down time with akurtakov (others welcome) to finalize the packaging wish-list.

Open discussion - <Your topic here>

Upcoming QA events

Action items

IRC transcript

jlaska #startmeeting Fedora QA Meeting 16:00
zodbot Meeting started Mon Feb 15 16:00:23 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00
jlaska #meetingname fedora-qa 16:00
zodbot The meeting name has been set to 'fedora-qa' 16:00
jlaska #topic Gathering 16:00
* kparal raises up his hand 16:00
adamw morning 16:00
jskladan hello 16:00
jlaska greetings folks 16:01
adamw Software Engineer Barbie also says hi 16:01
adamw http://www.engadget.com/2010/02/14/barbies-slides-into-the-cubical-becomes-a-computer-software-en/ 16:01
jlaska I believe maxamillion has a conflict today 16:01
jlaska finally, some mainstream recognition! :) 16:02
adamw looks like me on a good day! 16:02
jlaska adamw: does it come with more accessories (external monitors, bluetooth keyboard(s) etc...) 16:02
adamw this being barbie, all signs point to 'yes' 16:03
jlaska alright ... let's get started. hopefully wwoods, Viking-Ice and tk009 can join in as we go 16:04
jlaska #topic Previous Meeting Review 16:04
jlaska I've only captured 2 items from last week ... 16:04
jlaska #info jlaska will follow-up w/ nirik after the meeting to setup zodbot feed watchers 16:04
jlaska thanks nirik, we can check this off 16:05
jlaska we have zodbot notification for any additions to the F13Alpha, F13Beta and F13Blocker bugs 16:05
jlaska we might want to consider fedora-qa TRAC ticket additions as well 16:05
jlaska #info completed ... IRC zodbot notification for any additions to the F13Alpha, F13Beta and F13Blocker bugs (thanks nirik) 16:05
jlaska next up ... 16:05
jlaska #info jlaska to work w/ poelstra to give 'release validation' spin to quality milestones 16:06
jlaska This is completed as well ... I suggested a minor wording change (nothing crazy) 16:06
jlaska the test milestones are no longer specific to install 16:06
jlaska (e.g. Test Alpha 'Test Compose' and Test Alpha Candidate) 16:06
* Oxf13 is here, sorry 16:06
jlaska Oxf13: welcome! 16:07
jlaska #info completed, the wording around scheduled test milestones are no longer specific to install 16:07
jlaska that's all I had to track from last week ... anything I missed? 16:07
jlaska alright ... diving into the proposed agenda 16:08
jlaska #topic Rawhide Acceptance Test #3 16:08
jlaska Just a quick update on the last of 3 rawhide acceptance test (RAT) drops 16:08
jlaska you probably saw the announcement ... so I'm just summarizing here 16:09
jlaska #info RATS#3 test recap at http://lists.fedoraproject.org/pipermail/test/2010-February/088407.html 16:09
jlaska This milestone helped flesh out a few patches against rats_install 16:09
jlaska thanks wwoods for review feedback, I'll follow-up on those shortly 16:10
jlaska I need to update the RATS watcher script to watch the new URL (see ticket#115) ... I don't have immediate plans to look into that 16:10
jlaska so anyone with interest and time is welcome to pitch in 16:10
jlaska as always, Oxf13 thanks for coordinating the drops with dlehman 16:11
jlaska that's all I had on that ... let's dive into the Alpha ... 16:11
jlaska #topic Alpha TC test status 16:11
jlaska just a quick summary ... then onto some ideas raised by kparal and adamw 16:11
jlaska #info F-13-Alpha-TC2 is available for testing 16:12
jlaska #link https://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Install_Test_Results 16:12
jlaska #link https://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Desktop_Test_Results 16:12
adamw yay desktop testing 16:12
adamw wait, now I actually have to do some 16:13
jlaska woot! 16:13
adamw whose stupid idea was this? 16:13
jlaska adamw: deatils deatils 16:13
jlaska details too :) 16:13
jlaska so first note I have here was from adamw ... 16:13
jlaska #topic IDEA - publish live images along with 'test composes' 16:14
jlaska adamw: you raised this last week, I put this on the agenda so we wouldn't forget 16:14
* Viking-Ice sneaks in.. 16:15
jlaska Oxf13 I believe live images are provided for the release candidate drops (alpha, beta and final) ... with the new focus around desktop validation, are we able to provide them for 'test compose' as well? 16:15
jlaska Viking-Ice: welcome! 16:15
Oxf13 jlaska: well, we already make live images nightly 16:16
jlaska we've historically relied on the nightly images ... but depending on the state of the repo ... we can go days without any live images 16:16
Oxf13 duplicating that work would be, well, duplicated work 16:16
Oxf13 if the repo is busted for making the nightly images, how am I supposed to make them? 16:16
* jeff_hann here, sorry I'm late 16:16
* wwoods appears 16:16
adamw as I said in IRC, there's no need for a separate image generation process 16:16
jlaska wwoods: jeff_hann hey gang 16:16
adamw just take the nightlies from the appropriate day and stick them in the TC directory 16:16
adamw just want to have a single 'approved' live image that everyone's working from 16:17
adamw again, if it's too much work, it's not vital. 16:17
Oxf13 ... 16:17
jlaska yeah, doesn't matter to me which images we use ... just that we have images to test with 16:17
Oxf13 pointing to the live isos at the same time you point to the TC doesn't work? 16:17
adamw Oxf13: no, because anyone who reads it the next day can't use the same image 16:17
adamw but they can still get the right TC images 16:17
Oxf13 I suppose I could just make hardlinks 16:17
adamw Oxf13: the nightlies don't stick around, they get wiped and regenerated every day 16:18
jlaska would kparal's suggestion of always keeping around the previous successful live image build solve this? 16:18
Oxf13 hardlinks would solve this 16:18
adamw no, because you still have the problem if the next build is successful 16:18
adamw it's not that i'm worried a later build won't be 'successful' and no one will be able to test 16:18
adamw just worried that people would be testing with different stuff, leading to possibly confusing results... 16:19
adamw (tester A tests with 2010-02-15 and feature B is broken but feature C works, tester D tests with 2010-02-16 and gets the opposite result, world goes 'huh'?) 16:19
jlaska seems like a reasonable request 16:20
kparal nicely said :) 16:20
adamw Oxf13: yeah, a hardlink should do it I guess. 16:20
jlaska so the issue is making it clearer what people should be testing with ... instead of providing a URL to a directory that may or may not contain images? 16:20
Oxf13 well 16:20
adamw not just that. as I said, at present, it's not just an information issue 16:21
Oxf13 there is the problem of testing the static image instead of the next nightly 16:21
Oxf13 because the next nightly is what's going to be in the release 16:21
Oxf13 so testing what's older isn't all that useful, if the newer is broken 16:21
adamw Oxf13: but that applies equally to the non-live bits 16:21
Oxf13 adamw: except we don't generate the non-live bits every day 16:21
adamw the test compose isn't what's going to be in the release either 16:21
Oxf13 so we don't have the opportunity to test the newer bits 16:21
adamw still, it's difficult to do consistent testing and be sure about the results if different testers are testing different bits. 16:22
Oxf13 adamw: but we're still in "things change daily" 16:22
Oxf13 mode 16:22
Oxf13 we haven't stopped yet 16:22
Oxf13 these TC images aren't us stopping 16:22
Oxf13 they're just a snapshot of something we haven't yet produced this cycle to see what might be broken 16:23
Oxf13 in those specific things we don't always produce 16:23
jlaska so adamw, want to summarize what QA is requesting? 16:23
adamw i think we've wasted enough time already and should just move on 16:23
jlaska yup ... summarize, next steps and we'll move on 16:23
* jlaska suggests excessive use of #info 16:23
adamw if it takes this much debate to get it done it's probably not worth doing 16:23
adamw #info adamw would like a single set of nightlies to be associated with each TC build and kept available in that TC directory during TC testing 16:24
adamw #info oxf13 doesn't believe it's necessary 16:24
adamw next topic. 16:24
jlaska alrighty ... next up, I noted a topic from kparal. But kparal and Oxf13 may have discussed this already 16:25
Oxf13 more like counterproductive to testing the bits that will be in the release. 16:25
kparal I had almost the same question, this time about Alpha TC naming 16:25
jlaska Oxf13: it's something QA is asking for 16:25
jlaska I don't think it's counterproductive 16:25
jlaska we'll follow this up in a ticket after the meeting 16:25
jlaska #topic IDEA - Unique ISO image names for each drop 16:25
jlaska kparal: what's the latest 16:26
kparal well currently all TC iso's are named the same as the final Alpha iso 16:26
kparal Oxf13 replied: by not naming the isos etc the same for TC and the alpha release, we risk missing some bug in the behavior of Anaconda and the release name/number 16:26
jlaska you may have already discussed, but I wasn't clear on how the ISO file names are related to testing itself, was that cleared up? 16:27
kparal I had similar concerns as adamw, people would be testing something other than we expect. or they might suppose they already have the F13 Alpha final iso (it's called like that, isn't it?), but they won't 16:27
pjones kparal: Is there some reason to believe we're likely to have that kind of bug? 16:27
pjones considering that we change the names for every release and haven't been having issues with it... 16:28
jlaska we have people downloading the old ISO's a lot 16:28
kparal well, I wasn't sure why the names are not simply unique 16:28
jlaska normally, we have them validate the CHECKSUMS 16:28
Oxf13 not only that, but the TC stuff is on a completely different mirror system 16:29
kparal so the Oxf13's explanation is above. I didn't know about anaconda issues, then I suppose it makes more sense 16:29
Oxf13 we can't really protect against stupid people. 16:29
Oxf13 buildinstall does do things with version and product which does change anaconda behaviour 16:29
wwoods is there a TC# in the metadata anywhere? It'd be simplest to say "current test candidate is TC7" and then if .treeinfo says test-candidate = 6.. 16:30
Oxf13 of course, this is one of the dangers of opening pre-release test isos up to more people 16:30
wwoods but I don't really want to add new data if we don't have to. timestamps work, but people always get confused 16:30
Oxf13 you risk more people getting them who won't use them responsibly 16:30
Oxf13 wwoods: no 16:30
jlaska Oxf13: I don' think there is bad intent for the images 16:30
Oxf13 wwoods: it's at the top of the URL, but from that point down, by design, it looks exactly like the Alpha would 16:31
Oxf13 jlaska: I don't mean malintent, I mean ignorance 16:31
jlaska just looking for ways to make sure we are doing what's reasonable to make sure people know they are using the latest test images 16:31
jlaska okay 16:31
kparal I think we can move on 16:31
jlaska kparal: Oxf13: so in summary the filename used for the ISO image files is tightly coupled to the whole build process? 16:31
Oxf13 jlaska: yes 16:31
jlaska ah okay, not something I knew about 16:32
jlaska probably one of the new SOPs has this listed 16:32
jlaska https://fedoraproject.org/wiki/Composing_test_images ? 16:32
Oxf13 the input to the compose process drives the name of the iso, I don't manually make the isos and pick a new name each time 16:32
Oxf13 jlaska: probably Composing_Test_Composes once we make that 16:33
jlaska Oxf13: ah sweet, thanks 16:33
jlaska alrighty, kparal anything else to discuss here, otherwise we'll dive into AutoQA 16:33
Oxf13 test_images doesn't have any isos produced (beyond boot.iso) 16:33
kparal we can go on 16:34
jlaska alrighty 16:34
jlaska #topic AutoQA - depcheck update 16:34
jlaska wwoods: you have the mic ... any updates or milestones to share? 16:34
jlaska wwoods: we can come back to this later if you prefer? 16:36
jlaska alright ... moving on to resultsdb ... 16:37
jlaska #topic AutoQA - resultsdb discussion 16:37
jlaska kparal, wwoods or jskladan, anything to add here? 16:38
kparal well I would just mention that we had a small QA team discussion over phone 16:38
kparal and the results are available in autoqa-devel ML 16:38
* wwoods was afk a sec, deferring to kparal for the moment 16:38
jlaska #link https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000201.html 16:38
jlaska wwoods: okay 16:38
kparal we are currently studying other projects, how they define their results database, etc 16:39
kparal so any input can be sent to autoqa-devel 16:39
kparal that's all I suppose 16:40
jlaska kparal: okay, thanks for the update 16:40
jlaska #topic AutoQA - beakerlib 16:40
wwoods the current notion - if memory serves - is that we want to provide one unified resultsdb that al the autoqa tests can use to report (somewhat simplified) results 16:40
wwoods which will give us one place to view all test results - and ideally a few nice views of that data. 16:41
jlaska #topic AutoQA - resultsdb discussion 16:41
jlaska #info we want to provide one unified resultsdb that all the autoqa tests can use to report (somewhat simplified) results 16:42
jlaska one sec ... meeting tags 16:42
jlaska #info had a small QA team discussion over phone last week (results on autoqa-devel ML) 16:42
wwoods I think we had some example views we wanted to see - test results for a given package, test results for a given installable tree, all test results for package builds / updates for a given day, etc. 16:42
wwoods if you can think of a good use case you'd like - a specific view of the autoqa results data - please let us know 16:42
jlaska nice, thanks wwoods and kparal 16:43
wwoods further discussion will be on autoqa-devel. 16:43
wwoods oh, and as for depcheck: no progress, owing to the fact that I was out of the office and then sick all last week 16:43
jlaska status - plague :( 16:44
jlaska #topic AutoQA - beakerlib 16:44
jskladan ok, time for me to speak, i suppose :) 16:44
jlaska Adding a new topic here for the work jskladan and kparal have been looking into around beakerlib 16:44
jlaska any updates or problems you are experiencing? 16:44
jskladan well, idk if all af you are familiar with beakerlib 16:45
jlaska #link https://fedorahosted.org/beakerlib/ 16:45
jskladan its a set of libraries which RHEL guys use for writing automated tests 16:46
jskladan so we'd like to reuse those in out testing system (autoqa) 16:46
jskladan today kamil and me created a simple test, which demonstrates usage of beakerlib-based tests 16:47
jskladan #link http://jskladan.fedorapeople.org/beakerlib_helloworld.tar 16:47
jskladan it's nothing fancy - in fact, it only makes sure, that you have beakerlib installed, and then runs the test itself 16:47
jlaska nice, using the provided beakerlib vim syntax highlight! :) 16:47
jlaska rlAssertRpm "setup" 16:48
jlaska rlAssertExists "/etc/passwd" 16:48
jlaska rlAssertGrep "root" "/etc/passwd" 16:48
jlaska cool 16:48
jskladan and of course stores the report produced by beakerlib in the directory for autotest-lib result files 16:48
jskladan we'll be trying to assimilate other tests this week - probably init script testing stuff - and look into some more complex uses 16:49
jlaska this is great news, it's moving along faster than I had expected 16:50
jlaska nice work :) 16:50
jskladan so that's about it for now - if kamil does not have anything to add 16:50
jlaska #info today kamil and me created a simple test, which demonstrates usage of beakerlib-based tests 16:50
jlaska #info we'll be trying to assimilate other tests this week - probably init script testing stuff - and look into some more complex uses 16:50
kparal jskladan described it all 16:51
jlaska alright thanks for the update gang 16:51
jlaska I'll do a quick run through on the 2 remaining AutoQA topics 16:51
jlaska #topic AutoQA - install automation 16:51
jlaska #info Last week, Liam summarized current DVD automation efforts in an email to autoqa-devel 16:52
jlaska #link https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000199.html 16:52
jlaska Automating media (CD/DVD) installs using virt is funky b/c you have to do some strange stuff to provide your own boot arguments 16:52
jlaska so Liam outlined several techniques he's tried 16:53
jlaska including using dogtail to add custom boot args using virt-viewer 16:53
jlaska any feedback folks have on those proposed solutions would be good to have 16:53
jlaska once back from Chinese new year, he'll be working to integrate the best method into the current test 16:53
* Oxf13 has to bail, need to prepare for next appointment 16:53
jlaska and last up ... 16:54
jlaska #topic AutoQA - packaging 16:54
jlaska more small progress last week on identifying new build dependencies 16:54
jlaska it exploded to 50+ missing deps at one point, but akurtakov helped identify how to address that 16:54
jlaska This seems like the list of deps isn't every finalized :( 16:55
jlaska I'd really like to squash this so I can start making packaging progress 16:55
jlaska any ideas would be appreciated 16:55
jlaska My current plan, is to see if akurtakov (and others) can dedicate some time on IRC To help walk the list of missing deps once and for all 16:55
wwoods where's the work-in-progress stuff for people who want to help out? 16:56
jlaska #link https://fedoraproject.org/wiki/User:Jlaska/gwt 16:56
adamw i would suggest you make progress by starting packaging 16:56
adamw the only way you can ever be sure about the deps list is when you actually start packaging stuff and find out if it works 16:56
jlaska adamw: I'd like to as well, but I feel like I still don't know how long the packaging road is 16:57
jlaska I'm trying to find the leaf nodes to package ... and every time I look, I find another deps branch 16:57
jlaska so I'll revisit this week, and work towards adamw's suggestion ... just start packaging :) 16:58
jlaska #info current plan, is to see if akurtakov (and others) can dedicate some time on IRC To help walk the list of missing deps once and for all 16:58
jlaska #info I'll revisit this week, and work towards adamw's suggestion ... just start packaging 16:58
jlaska alright ... open discussion time 16:59
jlaska #topic Open discussion - <Your topic here> 16:59
jlaska anything not previously mentioned that we need to discuss today? 16:59
jlaska If no suggestions, I'll close out the meeting in 1 minute 17:00
adamw nup 17:01
jlaska alrighty ... thanks for the updates everyone 17:01
jlaska as usual, I'll send minutes to the list for review 17:01
jlaska #endmeeting 17:01

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!

Retrieved from "https://fedoraproject.org/w/index.php?title=QA/Meetings/20100215&oldid=152652"