From Fedora Project Wiki

Fedora Release Engineering Meeting :: Monday 2009-02-09

Package Signing--sha256

Automated QA

  • f13 created a script that can watch a set of repos (specifically the updates repos), detect when they change, and allow for execution of <something>
    • this would allow us to react to a new updates push and run whatever tests the QA team comes up with
  • wwoods working on RHTSlib stuff
  • continue discussion on #fedora-qa

Signing Server

  • trying to get test instance setup in PHX
  • jwb to help out with package signing in the future

Mass Rebuild

  • need to start on sooner rather than later
  • waiting for several features to solidify

IRC Transcript

f13 ping: jwb wwoods warren spot lmacken notting rdieter poelcat 10:01
* lmacken is here 10:01
* warren here 10:01
rdieter here 10:01
* jwb here 10:01
* wwoods here 10:02
f13 probably enough to get started 10:03
-!- f13 changed the topic of #fedora-meeting to: Fedora Releng - Alpha release wrapup 10:03
f13 So Alpha went out last week 10:04
f13 It was a very quiet and smooth release day. 10:04
f13 and from what I've seen, has been received fairly well 10:04
jwb rock on 10:04
f13 as promised, I wrote a lot of what I was doing down locally, and I need to slap it into a wiki page. It's going to be rough and will need some love, but it'll be something 10:05
f13 Any other thoughts on the Alpha? 10:06
f13 deep thinkers (: 10:06
jwb busy worrying about Beta 10:06
f13 heh 10:07
-!- f13 changed the topic of #fedora-meeting to: Fedora releng - Sha 256 sums 10:07
f13 http://fedoraproject.org/wiki/Features/StrongerHashes 10:07
f13 this hits releng in a couple places 10:07
f13 the hashes we produce with createrepo, and the SHA1SUM files we currently use for isos 10:07
* poelcat here 10:08
f13 the createrepo stuff is already taken care of in createrepo, and in pungi upstream 10:08
f13 the SHA1SUM file is a bit stickier. Obviously the file name is going to have to change 10:08
f13 and it wasn't too long ago that this file was MD5SUMS 10:08
jwb HASHFILE 10:08
f13 the thought this time is to name it more generically so that we can avoid future issues 10:08
f13 When i posted about it onlist, there was expressed some desire to have the filename also include information about what's inside of it 10:09
f13 I'm not so sure that I can do this cleanly/easily 10:09
jwb it seemed like a reasonable, if whiny, request 10:09
f13 yeah, the question becomes, how do I refer to the Fedora Fedora spin i386 CDs and DVD and netinst ? 10:10
f13 oh and insert an 11 in there somewhere. 10:10
f13 maybe just Fedora-11-Beta-i386-isos-HASHSUMS 10:11
f13 or 10:11
f13 Fedora-11-Beta-i386-isos-CHECKSUMS 10:11
jwb SUMSBITCHES! 10:12
f13 (: 10:12
jwb i like the CHECKSUMS one 10:12
f13 I think the latter is pretty easily done 10:12
f13 The live versions would be more like: Fedora-11-Beta-i686-Live{-$SPIN}-CHECKSUMS 10:13
f13 oh, lets drop the S off the end and just have it CHECKSUM 10:13
jwb yeah 10:13
nim-nim f13: as other said it would be much nicer if the checksum file format itself was something like 10:13
nim-nim algo:sum file instead of 10:13
nim-nim sum file 10:13
f13 nim-nim: I don't think it coudl then be used directly by sha256 10:13
f13 nim-nim: currently you can just do sha1sum -c <file> 10:14
f13 and ideally you'd be able to do sha256sum -c <file> as well 10:14
nim-nim f13: well, there needs to be veryfymychecksumdamnit command that just takes any checksum file in and does the right thing 10:14
f13 nim-nim: I think "needs" is a bit strong there. 10:14
nim-nim f13: otherwise it'll break again next format change 10:14
nim-nim f13: highly desirable then :p 10:15
f13 then I'm sure patches will show up quickly 10:15
nim-nim f13: sure :) 10:15
f13 writing our own software to validate isn't exactly the greatest ideal. 10:16
f13 idea. 10:16
f13 our files should be validatable on completely different distributions 10:16
f13 by standard tools 10:16
f13 but that's just my opinion 10:16
f13 and in the interest of getting this feature closer to 100%, I think I'll go the route of the CHECKSUM format and call it good 10:17
wwoods I think the file could have a generic name so long as we specify what algorithm is used in the file itself 10:17
nim-nim f13: you know just as well as me than having a fixed name but variable commands to check will jsut push the breakage into verify scripts 10:17
f13 wwoods: there could be a comment at the top of the file as to what algo is used 10:18
f13 that's not too hard to add to the code, a bit more difficult when creating the files manually like for the Live images 10:18
wwoods yeah that'd work. plus a stupid little wrapper script to read that and then exec $whichever_sum_program 10:18
f13 (but that could just mean that the live tools should be generating them as well) 10:18
f13 wwoods: I'll leave that script up to somebody else. 10:19
f13 Any further thoughts on stronger hashes? 10:20
poelcat add to the release notes? 10:21
f13 yeah, I mentioned above I'd poke the docs folks about this change 10:21
f13 and flag the bug itself that is open for this 10:21
f13 alright, moving along 10:22
-!- f13 changed the topic of #fedora-meeting to: Fedora releng - Automated QA 10:22
f13 I've made the first big step toward my promises at FUDCon 10:23
f13 I wrote up a script on Friday that can watch a set of repos (specifically the updates repos), detect when they change, and allow for execution of <something> 10:23
f13 this would allow us to react to a new updates push and run whatever tests the QA team comes up with 10:23
wwoods yay! 10:24
f13 it's a pretty simple script and should work both inside and outside of PHX 10:24
f13 I've also decided that the best use of my time is going to be coming up with the harnesses for detecting changes vs writing the actual tests themselves 10:24
wwoods definitely agree 10:24
wwoods all you need to do is write - and document - the hook 10:25
f13 I'm also going to poke Infra today and see about a F10~ instance to (ab)use for these things 10:25
wwoods so this will get run whenever *any* repo changes, or specifically updates-testing, or what? 10:25
f13 wwoods: the script I have now is specifically designed to monitor the updates repos 10:26
f13 f9-updates f9-updates-testing f10-updates f10-updates-testing 10:27
wwoods the basic idea is that the autoqa git repo will have a directory for each 'hook' 10:27
f13 it's just going to run in daemon mode, with a cache of repomd.xml for each repo, and it'll just diff them at a regular interval 10:27
wwoods and each dir will have a description of the expected environment / arguments for the scripts 10:27
wwoods and other folks can write scripts for that 10:27
f13 I haven't written anything yet in the script for once it detects a change 10:28
wwoods so in this case we've got a post-repo-update hook 10:28
wwoods so your harness could just assume that there's a dir full of scripts and run each of them with the same arguments and dump their output somewhere 10:28
wwoods probably I need to check on the RHTSlib stuff to see what standards we have for output and return values and such 10:29
f13 yeah 10:29
f13 I wasn't sure if we wanted the harness to handle the output, or each individual test, or if there was a top level script we coudl call from the harness that would handle all that 10:29
wwoods but yeah, please document the args and environment - like, say, maybe the scripts will get the path to an nfs-mounted copy of the repo? 10:29
f13 say if there was a top level 'run-tests' script for the post-repo directory 10:29
f13 the harness would call it, and wait for it to finish, that script would inturn call all the sub-scripts and bundle the results. 10:30
f13 wwoods: more like a http url 10:30
f13 or rather 10:30
wwoods hm. more likely we'll have a top-level 'run-tests' script that will be called as: run-tests $hookname $arg $arg ... 10:30
f13 they'll get a URI that is yum compatible? 10:30
f13 ok. 10:30
wwoods and it'll just run each script in the $hookname dir with the rest of the given args 10:30
f13 this is something we could probably discuss in #fedora-qa 10:30
wwoods whatever you like - just write up the definition so people know what to write scripts against 10:31
f13 cool 10:31
wwoods I'll work on RHTSLib info and the run-tests harness 10:31
f13 cool. 10:31
f13 I'm pretty excited about this, I think we should be able to have tests running in a short number of days 10:32
wwoods for the moment, just assume there's going to be a script named something like 'run-tests' that will act as I described it 10:32
f13 roger 10:32
f13 So I'm going to fine tune this script, and then explore other "buckets" that I can create a harness for easily 10:33
f13 Automated_QA_Testing_Project has a list of what we came up with at FUDCon 10:34
f13 I'll try to add links to sourcecode there as it gets checked into public repos 10:34
wwoods so, wait, post-repo-update is only going to get the URL of the newly-changed repo? 10:34
wwoods can you think of a way to also get the previous version of the repo, so we can run some diffs? 10:35
f13 that might be harder 10:35
f13 I think eventually it can be done, but I don't want to target that for version 0.1 10:35
wwoods okay, I added a 'post-repo-update' dir to autoqa 10:36
f13 Eventually we could rely on a message bus and for bodhi to drop on the bus "Hey I just composed this tag, to this path" and our monitoring script would already have in its history the previous path for that tag 10:36
wwoods please feel free to modify its README as you see fit. or just tell me how to change it. 10:36
f13 rgr 10:36
f13 Anything further on Auto QA? 10:37
f13 and releng's role in it? 10:37
f13 cool. 10:38
wwoods actually 10:38
f13 oh? 10:38
wwoods so one of the big goals that we've set for this year for fedora QA is making rawhide more usable 10:39
wwoods I feel silly saying that because that's pretty much our goal *every* year 10:39
wwoods but seriously. for real. 10:39
f13 we meanses it this time 10:39
wwoods anyway we're taking a look at the rawhide process and trying to figure out good places to insert tests and ways to streamline it 10:39
wwoods so people get fixes faster and there's less "oh rawhide is broken" time 10:40
wwoods so this is a general heads-up. 10:40
f13 rawhide is my next 'bucket' to look at 10:40
wwoods more specific requests will follow, I'm sure 10:40
wwoods right now we're just trying to figure out stuff. like why mashing takes so damn long. 10:41
f13 but it's likely to be the same style as updates, watching a static location for repodata change and calling off a set of tests. 10:41
f13 wwoods: mash is far far far faster than it was months ago 10:41
f13 we're down to about 2~ hours to make rawhide 10:41
wwoods right, but adamw points out that pretty much every other distro has their devel branch update as builds are made 10:42
wwoods rather than nightly 10:42
f13 maybe 2:25 from cron start to bits on the master mirror. 10:42
f13 wwoods: how are they possibly keeping mirrors in sync? 10:42
f13 and do these other distros do multilib? 10:42
wwoods it's 2:15 to make rawhide, and 1:45 of that is mash. 10:43
jwb continuous update sounds wrong-ish 10:43
wwoods the other :30 is, basically, pungi 10:43
jwb our buildroots do that, but not the repo 10:43
f13 If we didn't have multilib, and we didn't purge older builds from the repos, we could have our rawhide repos update with new content in about 5 minutes after every build 10:44
wwoods f13: I'm fuzzy on whether mandriva did multilib the way we think of it 10:44
f13 of course getting mirrors to be able to process our tree that fast would be a joke 10:44
wwoods and as for mirrors, I think they had less trouble keeping mirrors in sync because they didn't have to sync it as far (or to as many) to get it to their testers 10:44
wwoods i.e. I think it was feasible for their testers to all use the Master Mirror (or a mirror a single hop away) 10:45
wwoods like I said, we're still analyzing stuff 10:45
f13 they also likely had far far less on the mirror 10:45
wwoods also true 10:45
f13 our mirror tree is a bit... bloated 10:45
f13 as is our package set 10:45
wwoods but then we don't really need to sync everything that was in Extras more than daily 10:45
f13 wwoods: so in theory, we do have repos that are continuously updated throughout the day. But it would mean forcing everybody to use PHX, and not having multilib 10:46
f13 wwoods: Extras is empty, but yes 10:46
wwoods it's just old Core that would be most useful to have updating continuously 10:46
f13 but the more complicated you make our sync suggestions, the less likely it'll be that mirrors do it right. 10:46
f13 but it's good that you're looking into it 10:47
wwoods everything worth doing is hard. 10:47
f13 I'll try to help with as much information as I can 10:47
wwoods but then, plenty of things *not* worth doing are also hard 10:47
wwoods so we gotta be sure we're doing the right thing. 10:47
wwoods sometimes I feel like "not using multilib" isn't a huge problem 10:47
wwoods but anyway 10:48
wwoods research phase, actionable stuff to come, hold onto yr butts and brace for a lot of stupid questions 10:48
f13 sometimes I think "stop supporting multilib" would solve a /lot/ of our problems. 10:48
f13 well, more than sometimes. 10:49
wwoods faster RPM dependency solving, smaller repos, cut rawhide building and updates pushes down to a 30-minute process 10:49
f13 kill a ton of conflicts 10:49
* f13 sighs 10:50
f13 wwoods: thanks for the info, we'll brace ourselves for the questions and be open minded 10:50
-!- f13 changed the topic of #fedora-meeting to: Fedora releng - Signing Server 10:50
f13 so yeah, that magical signing server thing 10:50
jwb uh huh 10:51
f13 I've asked jwb to help out here and get a test instance or 2 setup in PHX so that we can further evaluate it in its natural enviroment. 10:51
jwb yep 10:51
jwb i need to file a ticket 10:51
f13 since I've been focused on alpha and QA, I'm going to try this delegation thing and see how that works out (: 10:51
jwb was derailed by trying to learn about signing as it is today, and the fullfilelist thing 10:51
f13 nod 10:51
f13 that's been quite helpful too 10:51
jwb will be when i get it done anyway :) 10:52
f13 heh 10:52
jwb need to figure out the sudo ftpuser thing and that should be it 10:52
f13 I talked to mjcox about the usb gpg device he was using, but he told me it doesn't support keys as big as we need and it would be pretty slow 10:53
f13 so I think we'll just use software 10:53
f13 As jwb mentioned, I've also started training him on doing signings as they stand today 10:53
f13 It is just silly to rely upon a single person to do all that 10:53
f13 and I'm really confident that we can trust jwb with the F9/10 keys 10:54
f13 so with luck we'll have another signer in the near future for pushing updates 10:54
f13 anybody have any objections / thoughts on this? 10:55
f13 (would be nice to have notting here right now) 10:55
f13 I'll take silence as consent. 10:56
-!- f13 changed the topic of #fedora-meeting to: Fedora releng - Mass Rebuilds 10:56
f13 last topic of today 10:56
* wwoods consents! 10:56
f13 We've got potential for mass rebuilds on the horizon 10:56
rdieter kudos to jwb, may ye not get drunk with power 10:56
f13 GCC hit rawhide, needs a bit of time to stew. 10:56
jwb rdieter, thanks. promise i wont :) 10:56
f13 rpm for 256 checksums, I'm not sure the status of that 10:57
f13 rpm for font provides, close, but not there yet 10:57
f13 Feature freeze is coming up soon, we should get these rebuilds done prior to feature freeze 10:57
f13 which means we need to start them very soon 10:57
jwb is it automated, or maintainer driven? 10:57
f13 since we're going to do /every/ package, we should probably try to automate it 10:58
f13 time is short and waiting sucks 10:59
jwb aye 10:59
f13 so whomever is in FESCo, please start pushing hard to get some dates on these items 11:00
jwb i'll propose we start monday, assuming all the pieces are in place? 11:01
f13 Sure, I'm not doing much next week (: 11:02
-!- f13 changed the topic of #fedora-meeting to: Fedora releng - Open Floor 11:04
f13 any open topics? 11:04
f13 (I noticed that poelcat filed a ticket looking for a new meeting minder) 11:04
poelcat https://fedorahosted.org/rel-eng/ticket/1275 11:05
poelcat :) 11:05
f13 alright, time to wrap up, lots to do! 11:11
f13 thanks all, and thanks poelcat for the time and effort you've continued to put into this team. 11:11

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!

Retrieved from "https://fedoraproject.org/w/index.php?title=ReleaseEngineering/Meetings/2009-feb-09&oldid=472402"