From Fedora Project Wiki
Fedora Release Engineering Meeting :: Monday 2008-02-25
Rebuild status
- going fairly smoothly
- the triage team is taking good swaths of those and isolating the build failures and sorting them into buckets (either compile failure, or other failure)
Orphan package removals
- propose those orphans that haven't been picked up be blocked from dist-f9 today
- gives some folks time to realize "oh that was important, I'll take it over" before we hit beta freeze
- warren will work on analyzing the orphan list
Release Engineering Representative
Discussion of Custom Spins
- 11 more in the pipeline
- See IRC log
Misc
- setting up a meeting tomorrow to talk about the fedora build system instability and get some time lines for fixing what we can, etc.
IRC Transcript
| -!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering Meeting - Roll Call | 12:59 | |
| f13 | ping notting jeremy poelcat jwb spot warren poelcat rdieter wwoods lmacken | 13:00 |
|---|---|---|
| * warren here | 13:00 | |
| * jeremy | 13:00 | |
| jwb | here | 13:00 |
| * wwoods nom nom granola nom nom | 13:01 | |
| * poelcat here | 13:02 | |
| * notting is here | 13:02 | |
| f13 | ok, we've got a few things to discuss so lets get started. | 13:02 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering - Rebuild status | 13:02 | |
| jwb | that went surprisingly well | 13:03 |
| f13 | All the first round of builds completed on Friday | 13:03 |
| f13 | early Friday morning in fact | 13:03 |
| f13 | and all the bugs for the failures were filed later that day | 13:03 |
| f13 | the triage team is taking good swaths of those and isloating the build faiulre and sorting them into buckets (either compile failure, or other failure) | 13:04 |
| f13 | lots of fixes have been happening, and most of the buildsystem generated failures have been resubmitted | 13:04 |
| f13 | all in all this has been one of the smoother rebuilds from the submission and followup side of things. | 13:04 |
| f13 | I want to clean up the scripts I used for more general purpose and check them into the rel-eng git tree | 13:04 |
| f13 | GCC 4.3 rebuilds have been a "would be nice" sort of feature so I'm not going to sweat it too much if things aren't rebuilt by the beta or even the release. | 13:05 |
| f13 | (unless of course the failure to rebuild results in a failure to fix some other bug or blocker) | 13:05 |
| jwb | related topic, did you see someone's request for a package for the bump script? | 13:06 |
| f13 | yeah, I'm fine with it going into a package. | 13:06 |
| f13 | probably makes sense in rpmdevtools | 13:06 |
| f13 | although it is pretty Fedora specific | 13:06 |
| wwoods | fedora-packager? | 13:06 |
| jwb | something like that, yeah | 13:07 |
| f13 | wwoods: we have other pretty Fedora specific things in rpmdevtools so I don't think it really matters much | 13:07 |
| wwoods | makes sense | 13:07 |
| jwb | overall i don't think many of the other rel-eng scripts should really be "packaged" | 13:07 |
| f13 | nod | 13:07 |
| jwb | ok, sorry to derail | 13:07 |
| f13 | typically they're mostly references to start from in order to accomplish your specific task | 13:07 |
| f13 | and that doesn't make much sense as a package outside of examples/ | 13:08 |
| f13 | anywho, the fixes will continue, and folks like jeremy have offered to look at the real hard compile issues. | 13:08 |
| jwb | jakub seems to be poking at issues that get reported to -devel | 13:09 |
| f13 | gcc43errors is the bug alias that the triage folks are putting the actual compile issue bugs on | 13:09 |
| f13 | I think at this point, releng can just let this work itself out naturally | 13:09 |
| f13 | moving on... | 13:10 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering - Orphan package removals | 13:10 | |
| f13 | warren: you were running with this I think, what's the current status? | 13:10 |
| warren | f13, there were relatively few and many were picked up | 13:11 |
| warren | f13, come back me in about 10 minutes, i'll check it now | 13:11 |
| f13 | warren: ok. | 13:11 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering - Making releases a snap | 13:11 | |
| f13 | stickster has called for folks to report to him on SOPs for doing releases | 13:11 |
| f13 | and we owe him a report | 13:11 |
| jwb | i'm sort of confused as to what he wants | 13:12 |
| f13 | let me pull up the archived mail | 13:12 |
| poelcat | f13: he wants a point person | 13:12 |
| f13 | https://www.redhat.com/archives/fedora-advisory-board/2008-February/msg00136.html | 13:12 |
| poelcat | so we can have a concall in advance of the beta to make sure all the groups are in sync | 13:12 |
| poelcat | and dependencies are handled in order and in time | 13:13 |
| f13 | so this is just for beta this time, and we'll do it again for PR and Final? | 13:13 |
| jwb | hm | 13:13 |
| jwb | is he going to include arch teams? | 13:13 |
| f13 | that's a fun question | 13:14 |
| f13 | theoretically secondary arch teams move at their own schedule | 13:14 |
| f13 | and aren't tied directly to our release dates | 13:14 |
| poelcat | in theory we do it for all of the releases... and each time it gets faster and more organized :) | 13:14 |
| jwb | i nominate f13 | 13:14 |
| f13 | So unless somebody else wants to volunteer, I'll handle this | 13:14 |
| f13 | gee thanks jwb (: | 13:15 |
| jwb | does anyone else really make sense? | 13:15 |
| f13 | probably not the first time around | 13:15 |
| wwoods | you *are* kind of the guy who turns the handcrank | 13:15 |
| jwb | if you'd like, i can attend as well if possible | 13:15 |
| wwoods | I guess the theory is that we can take notes and then, eventually, other people will know how the sausage gets made | 13:15 |
| jwb | though it would be more of a "listen in" kind of thing | 13:16 |
| warren | f13, I can't login to the database to do a query | 13:16 |
| f13 | warren: I saw. | 13:16 |
| f13 | ok, I'll take it this time. | 13:16 |
| * f13 adds another note to the todo list | 13:16 | |
| -!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering Orphan package removals | 13:17 | |
| f13 | lets get back to this one for a moment. | 13:17 |
| f13 | since we're nearing very close to the Beta freeze, I propose those orphans that haven't been picked up be blocked from dist-f9 this week, today. | 13:17 |
| f13 | that'll give some folks time to realize "oh that was important, I'll take it over" before we hit beta freeze | 13:17 |
| jwb | agreed | 13:18 |
| f13 | warren: that seem sane to you? | 13:18 |
| notting | seems ok | 13:18 |
| warren | umm | 13:19 |
| warren | the orphan list seems to have grown since last week | 13:19 |
| warren | f13, can we warn with a fresh list later today with thursday the target removal day? | 13:20 |
| warren | f13, I need a little time to analyze the new list | 13:20 |
| * warren not sure if his SQL query is correct | 13:20 | |
| f13 | warren: sure, that's fine. This week though. I don't want to rip off the bandaid and immediately freeze the tree | 13:20 |
| warren | f13, nod | 13:21 |
| warren | f13, the freeze next week is a hard freeze? | 13:21 |
| warren | f13, or a slushy one like F8's? | 13:21 |
| f13 | the freeze on the 4th is | 13:21 |
| f13 | uh | 13:21 |
| warren | honor system or manual tagging? | 13:21 |
| f13 | what is your definition of 'slushy' ? | 13:21 |
| warren | Remember last time around we used honor system? | 13:21 |
| f13 | we did? | 13:21 |
| warren | Maybe that's the wrong word | 13:22 |
| f13 | we branched and told people if they really want to build for F8, build from the branch | 13:22 |
| warren | huh? | 13:22 |
| f13 | otherwise if they built from devel it would go to dist-f9 | 13:22 |
| warren | isn't branching optional at freeze point? | 13:22 |
| f13 | warren: I think we're talking about two different freeze points | 13:23 |
| warren | grr, can't load the wiki | 13:23 |
| warren | f13, I thought next week's freeze is for Beta and feature freeze, and the wiki said people can optionally branch CVS at that point. | 13:23 |
| f13 | regardless, the freeze on the 4th is hard, you have to request tags from releng | 13:23 |
| warren | That sounds fine, but when was that decided? | 13:24 |
| f13 | http://fedoraproject.org/wiki/ReleaseEngineering/Overview#head-7dfffd8d508d35c931d4b7cc519fce1b00019e4c | 13:24 |
| warren | I thought the F8's version of this "freeze" worked out extremely well compared to previous freezes that operated in a manner similar to what you are saying happens next week. | 13:24 |
| f13 | warren: F8's only works if we allow for branching | 13:25 |
| f13 | and it really only works at the tail end of a development cycle, not in the middle of it | 13:25 |
| warren | when do we allow branching this time? | 13:25 |
| f13 | we never decided at what point between beta release and final freeze we'd start allowing for branches. | 13:25 |
| warren | I thought the schedule said at beta freeze point | 13:26 |
| warren | I could be wrong, but I can't load it to find out | 13:26 |
| warren | oh, http://fedoraproject.org/wiki/Releases/9/Schedule March 13th allow branching | 13:26 |
| warren | nm | 13:26 |
| f13 | Overview still says: | 13:26 |
| f13 | Early (optional) CVS branching | 13:26 |
| f13 | Right after Beta release or sometime between Beta and Pre-release, we allow for early branching of software. This allows developers to check in new features and otherwise unstable changes that would not be suitable to introduce to the current release. | 13:26 |
| f13 | but I guess the schedule took that as 'right after beta release' | 13:27 |
| f13 | but in reality that's going to be very very early for a lot of software | 13:27 |
| warren | OK, sounds good. | 13:27 |
| * warren works on analyzing the orphan lits | 13:27 | |
| f13 | ok, moving on. | 13:28 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering - Jeremy's spin writeup | 13:28 | |
| f13 | shamefully, I haven't taken the time to read this fully through yet. | 13:28 |
| f13 | jwb: have you? | 13:28 |
| jwb | f13, i told jeremy i would reply today. i still intend to | 13:28 |
| jeremy | it'd be nice to get some feedback so that I can post it for more people to see | 13:28 |
| jeremy | so I can get spoleeba off my back :-P | 13:28 |
| f13 | yeah. | 13:28 |
| jwb | jeremy, you'll have something this afternoon from me | 13:29 |
| jeremy | jwb: sounds good, thanks | 13:29 |
| f13 | I really wanted to get to it before the meeting, finding out why rawhide fell over took too much time though :/ | 13:29 |
| f13 | I will also get you feedback today, soon even, so that you can send it out today | 13:29 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering - Open Discussion | 13:29 | |
| jeremy | sounds like a plan | 13:29 |
| jwb | speaking of spins | 13:30 |
| jwb | i've done two now for mether. xfce and hindi | 13:30 |
| * f13 gives jwb the floor | 13:30 | |
| jwb | he has like 4 more queued up (or some number) | 13:30 |
| jwb | all localized | 13:30 |
| warren | oh damn | 13:30 |
| warren | the orphan list has grown | 13:30 |
| warren | 125 packages now | 13:30 |
| jwb | i started a SOP for doing spins, but it's empty now. i intend to fill it out with this next one | 13:31 |
| notting | jwb: only 4? | 13:31 |
| jwb | no, more. | 13:31 |
| * jwb counts | 13:31 | |
| warren | oh wait, some of these were already removed | 13:31 |
| jwb | 11 more | 13:31 |
| f13 | I got a scary message from spevack today, where he wants to do a localized spin for wherever we have an event | 13:31 |
| f13 | that's going to lead to a /lot/ of spins, just for lang selections | 13:31 |
| f13 | I for one, don't like this | 13:31 |
| jwb | we have 12 total right now. 1 done, 11 to go | 13:31 |
| jwb | just for .in | 13:31 |
| jwb | (i think) | 13:32 |
| f13 | it's a lot of duplicated content, it's a lot of releases to try and keep track of the sources of for GPL reasons, it's a lot of QA to go through, and it'll start making our download pages rather cumbersome | 13:32 |
| mether | one of them is for Srilanka | 13:32 |
| wwoods | ugh. I'm gonna need a team of hax0rs to work on automating testing live images if we're going to do this much stuff | 13:32 |
| jwb | this is an illustrative example of why we need to review jeremy's doc | 13:33 |
| jwb | because the people in this meeting will not scale to test all of these, nor _can_ we in some cases | 13:33 |
| jwb | i don't speak/read hindi, etc | 13:33 |
| f13 | right. I think we said we wanted to see interesting things folks have done with spins, and I'm just not sure of "only include my local language" is all that... interesting. | 13:34 |
| f13 | or rather, we wanted to /host/ interesting things | 13:34 |
| notting | f13: then we need to fix livecd-creator | 13:34 |
| jwb | notting, ? | 13:34 |
| wwoods | jwb: right - there needs to be docs and tools for letting the *spinners* test their spins | 13:34 |
| notting | jwb: multiple langauges, selectable | 13:34 |
| wwoods | any plan that involves dumping more work on rel-eng or QA = fail | 13:34 |
| f13 | notting: wouldn't that be gdm in F9? | 13:35 |
| notting | f13: ??? GDM in *F8* does that | 13:35 |
| jwb | personally, i have no issues mashing buttons to churn out isos and throw them on spins.fp.o | 13:35 |
| jeremy | wouldn't that be that gdm has allowed it for a long long time | 13:35 |
| f13 | ok, so I'm confused | 13:35 |
| notting | these 20 different spins a) set a default timezone and /etc/sysconfig/i18n LANG b) occasionally adjust the translation sset | 13:36 |
| pingou | ! | 13:36 |
| f13 | jwb: yeah, I just have issues with those carrying the Fedora name/logo without anybody doing QA on them, ensuring that they actually install, that no regressions creeped in due to using newer code sets, finding a place to stash the srpms used (!!) for GPL reasons, etc... | 13:36 |
| f13 | pingou: that's not necessary in this meeting, just blurt it out. | 13:36 |
| pingou | there is the language but also the keyboard that is interesting to changes between the different livecd | 13:37 |
| notting | pingou: i stand by my statement - 10-20 livecd images that only differ by 1% is wasteful | 13:37 |
| f13 | seems that creating another 700~ meg download for that is.... silly | 13:37 |
| wwoods | unless we work out some magic jigdo way of storing them | 13:38 |
| jwb | f13, i'm not disagreeing. i think that's where we need to go back to the board and say "please clarify why these are official spins" | 13:38 |
| pingou | what about using the local mirror ? | 13:38 |
| f13 | pingou: in many cases "what local mirror?" | 13:38 |
| jeremy | notting: xdelta! | 13:38 |
| pingou | f13, something like storing the french version on the french mirror, the dutch on the dutch... | 13:39 |
| warren | jeremy, will xdelta work when the changes are within the compressed squashfs image? | 13:39 |
| jwb | pingou, if they have a local mirror, why can't they do the spin themselves? | 13:39 |
| pingou | jwb, they start to do it | 13:39 |
| f13 | pingou: we don't have a 1:1 mirror:country relationship | 13:39 |
| jwb | we could add links to those | 13:39 |
| jeremy | warren: dunno. someone is welcome to try :-) | 13:39 |
| warren | jeremy, the pyjigdo guys were talking about making jigdo uncompress and recompress squashfs images with jigdo... | 13:40 |
| jwb | i don't think we're going to make a huge amount of progress on this today. i just wanted to point at where i am with the spins stuff and raise the bigger question | 13:40 |
| * jeremy sticks his fingers in his ears and goes "la la la la la la la la" | 13:40 | |
| notting | warren: that sounds fraught with failure | 13:40 |
| warren | I just mentioned it | 13:41 |
| warren | ok... so the orphan list has definitely grown since last week | 13:41 |
| * f13 has yet to see pyjigdo in action, for it's original purpose, let along feature creep | 13:41 | |
| f13 | er alone. | 13:41 |
| f13 | jwb: thanks, that does help | 13:42 |
| warren | looks like I'll have a complete list of orphans in about 30minutes hwen te script finishes | 13:42 |
| warren | (koji is really slow now) | 13:42 |
| warren | f13, really, there is no need to be so down on them, they are free to work on whatever they want. | 13:42 |
| mbonnet | warren: it's working fine for me, seems snappy, and neither the database nor app server are loaded | 13:42 |
| warren | mm | 13:42 |
| mbonnet | warren: sure you're not having connection issues? | 13:43 |
| warren | mbonnet, abadger1999 mentioned having issues coming in through RH VP | 13:43 |
| warren | mbonnet, RH VPN | 13:43 |
| jwb | f13, i've used pyjidgo. it worked fairly well | 13:43 |
| warren | mbonnet, I seem to as well | 13:43 |
| jwb | warren, why are you talking to koji? shouldn't orphans be done from pkgdb? | 13:43 |
| f13 | warren: I'm not being down, it's just we're a week away from the beta freeze and I haven't seen a single patch for being able to do jigdo stuff for F9. | 13:44 |
| warren | jwb, pkgdb knows about orphaned packages, but not if they were already removed | 13:44 |
| jwb | ah | 13:44 |
| jwb | nor deltarpms... jeremy ? | 13:44 |
| warren | abadger1999 and jdieter were working on deltarpms? | 13:44 |
| warren | oh wait | 13:44 |
| warren | no | 13:44 |
| jeremy | jwb: I haven't had anything to do with any of the work for deltarpms this time around | 13:44 |
| warren | lmacken | 13:44 |
| jwb | k, sorry | 13:45 |
| warren | jwb, most of the deltarpm work is needed in bodhi, with a little rel-eng scripting after that | 13:45 |
| abadger1999 | warren: Signing me up for extra work again? ;-) | 13:45 |
| warren | abadger1999, maybe. | 13:45 |
| jeremy | abadger1999: we try! :) | 13:45 |
| f13 | sadly nothing tends ot be 'little' when it comes to releng scripting | 13:45 |
| jwb | warren, so we'd only target updates with that? | 13:45 |
| warren | jwb, that's where most of its usefulness is no? | 13:46 |
| warren | f13, relatively little compared to the bodhi work | 13:46 |
| jwb | yes. i was just checking. if so, it's fairly disjoint from being tied to F9 (or any other release) | 13:47 |
| f13 | except that unleashing deltas upon a stable release without any testing during rawhide sounds like a fun event. | 13:47 |
| warren | grr, you're right | 13:48 |
| mether | FYI SUSE has no build system integration and just runs createdeltarpm or whatever post update generation | 13:48 |
| f13 | mether: they also don't have a very public mirror system and are able to control that a lot better | 13:48 |
| notting | mether: um, that would still be 'deltarpm work needed in bodhi' | 13:48 |
| warren | f13, would you accept adding the deltas to rawhide after beta is released? Only thing we're missing is a defined place to put them in the tree right? | 13:49 |
| mether | notting: they run a cron job or something async | 13:49 |
| f13 | warren: honestly I don't know what all is missing, I haven't talked to anybody about it in a while. Rather, nobody has talked to /me/ about it in a while. | 13:49 |
| notting | isn't that what this meeting is for? | 13:49 |
| warren | yes | 13:49 |
| f13 | notting: you'd think that. | 13:49 |
| f13 | who's the feature owner? | 13:50 |
| warren | OK, it seems to me that the only thing missing to add deltas to rawhide is a defined place to put it in the tree? | 13:50 |
| warren | sort of me | 13:50 |
| warren | jdieter has been taking point | 13:50 |
| f13 | warren: so then we have patches to mash to generate deltas, and huristics on what to generate deltas against, and a way of dealing with signed packages vs unsigned and... | 13:50 |
| warren | f13, is mash the right place to put it? | 13:51 |
| f13 | warren: mash is what creates rawhide | 13:51 |
| f13 | or a cron job does really | 13:51 |
| f13 | so patches to the cron would also work I suppose. | 13:51 |
| warren | what's the source repo for this? | 13:51 |
| warren | I'll point this at jdieter | 13:52 |
| warren | I can work on this after feature freeze myself, but not now. | 13:52 |
| f13 | git.fedorahosted.org/git/releng | 13:52 |
| f13 | I'm not going to be too keen on working on this after feature freeze though. I really really want to land F9 on time and I don't want to make things more unstable in the rawhide creation land | 13:52 |
| warren | The sad thing about this is, there is no reason why deltas absolutely need to be done during tree creation. | 13:53 |
| warren | they can be async | 13:53 |
| warren | it does make it crappy for mirroring though | 13:53 |
| warren | I'll talk with jdieter | 13:53 |
| warren | I'm afraid this will slow down rawhide creation | 13:54 |
| f13 | yeah, our mirroring system is fragile enough as it is. | 13:54 |
| warren | ah, orphan list is complete | 13:54 |
| warren | http://paste.ausil.us/284 | 13:55 |
| warren | I'll send a warning to the lists that these packages and anthing that depends on them will be removed Thursday afternoon if they are not owned. | 13:55 |
| warren | ok? | 13:55 |
| jwb | xmlrpc... | 13:55 |
| jwb | that seems... important | 13:55 |
| warren | I DID warn people last week | 13:56 |
| f13 | warren: good. | 13:56 |
| * warren sending | 13:56 | |
| f13 | anything else this week? | 13:57 |
| jwb | yah | 13:57 |
| jwb | what about the meeting you wanted to setup with the mikes and mark on koji? | 13:57 |
| jwb | or is that outside the scope of this meeting? | 13:58 |
| f13 | kinda outside I think. | 13:59 |
| jwb | k | 13:59 |
| f13 | but for those listening at home, we're setting up a meeting tomorrow to talk about the fedora buildsystem instability and get some timelines for fixing what we can, etc. etc.. | 14:00 |
| f13 | Ok, calling it, thanks folks. | 14:01 |
Generated by irclog2html.py 2.3 by Marius Gedminas - find it at mg.pov.lt!