From Fedora Project Wiki
Fedora Release Engineering Meeting :: Monday 2008-11-24
Fedora 10
- In good shape for release of Fedora 10 tomorrow
- Fedora 10 zero day updates are ready to go
- f13: one thing I was toying with is when we branch and freeze, making the development/ path continue on with say F12 content, and start publishing the F11 content to releases/11/Everything/
Fedora 11 Schedule
- need to make sure F12 branching is visible
- finding ways to reduce package churn
- QA team monitoring status of features during release
- proposal at http://poelstra.fedorapeople.org/schedules/f-11/f-11-standard.html
- poelcat to make adjustments to proposed schedule and republish
- move alpha freeze forward one week
- move alpha release forward one week
- remove critical package freeze from schedule
- offset feature freeze and beta freeze by one week
- vote on schedule to present to FESCo next week
IRC Transcript
| f13 | ping: notting jeremy rdieter wwoods lmacken warren spot | 10:00 |
|---|---|---|
| jeremy | hi | 10:00 |
| f13 | poelcat: ping | 10:00 |
| * lmacken | 10:00 | |
| * poelcat here | 10:01 | |
| notting | sorry, here now | 10:01 |
| * spot is here | 10:01 | |
| wwoods | zoom! ding ding. | 10:02 |
| f13 | whee. | 10:04 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora releng - Fedora 10 | 10:04 | |
| f13 | so there is this thing we gotta do tomorrow. | 10:04 |
| f13 | no big thing, just take few moments of our time | 10:05 |
| f13 | releasing Fedora 10 | 10:05 |
| wwoods | drive to alabama for thanksgiving with our wife's family? | 10:05 |
| wwoods | oh, maybe that's just me | 10:05 |
| f13 | wwoods: yes, that's it | 10:05 |
| f13 | We're in really good shape | 10:05 |
| wwoods | awesome. hope you guys like deep-fried cajun turkey | 10:05 |
| f13 | all our bits are very well synced to a number of mirrors, we've got 0-day F10 updates in place and about to be public via mirrormanager, skvidal got the bits synced over to the torrent box and torrents made up, and we've got some pre-seeding of torrents setup too | 10:06 |
| f13 | the key peices left are a tested release webpage set, a release announcement, and me pulling the trigger early tomorrow | 10:06 |
| f13 | and then it's on to turkey and drinking | 10:07 |
| f13 | oh and F11 | 10:08 |
| notting | who is doing the release announcement? | 10:08 |
| f13 | notting: the docs team usually has a release announcement made, and I send it out | 10:08 |
| wwoods | f13: don't forget flipping the bits in releases.txt for preupgraders | 10:08 |
| f13 | the web team preps the pages for release | 10:08 |
| f13 | wwoods: yep, that's a current filed ticket in the Fedora 10 Final tracker in trac | 10:08 |
| wwoods | (I know, there's already a ticket for that, just mentioning it with the other bits) | 10:08 |
| f13 | er milestone | 10:08 |
| f13 | I was going to flip the rawhide bit yesterday but wound up taking a much much needed night away from the kid on a date with my wife | 10:09 |
| f13 | and spending part of that on the computer would have been not good for my health | 10:09 |
| wwoods | yeah, we all run pretty close to computer-burnout-redline around release time | 10:09 |
| f13 | and since we don't want rawhide sync getting in the way of our permission bit flip, I'm not going to reset rawhide today either, I'll do it tomorrow, for Wed's rawhide. | 10:10 |
| f13 | This is one aspect of the transition we should think about for F11 | 10:10 |
| nirik | is there a new fedora-release waiting in rawhide ? | 10:11 |
| jeremy | f13: that sounds reasonable to me | 10:11 |
| f13 | nirik: yeah, there has been one in dist-f11 for quite a while. | 10:11 |
| notting | Update 29 Package(s) | 10:11 |
| notting | Total download size: 59 M | 10:11 |
| notting | not bad for day-0 | 10:11 |
| f13 | nirik: a new fedora-release is needed whenever we create a new build target so it has been there. | 10:11 |
| f13 | notting: I think there were close to 300 updates for F10 | 10:11 |
| * nirik nods. Sounds good. | 10:11 | |
| notting | f13: right, but not all of them applicable to everyone | 10:12 |
| f13 | right | 10:12 |
| f13 | 327 now to be exact. | 10:12 |
| spot | since most of them were mine, they're applicable to almost no one. ;) | 10:12 |
| notting | f13: what's your thoughts on the rawhide transition? earlier? | 10:12 |
| f13 | notting: yeah. | 10:12 |
| notting | (since there is no later) | 10:12 |
| f13 | notting: one thing I was toying with is when we branch and freeze, making the development/ path continue on with say F12 content, and start publishing the F11 content to releases/11/Everything/ | 10:13 |
| f13 | we'd be doing 2 composes a day | 10:13 |
| f13 | but we let the next rawhide move on, and we start getting things synced up in the releases/ tree | 10:13 |
| notting | hm. could work, but could also confuse people about the final release | 10:13 |
| wwoods | If I remember right, we were talking about doing the rawhide transition *at* Preview | 10:13 |
| f13 | notting: yeah, there is certainly going to be confusion | 10:13 |
| f13 | notting: but there is also confusion now where there is "rawhide" which is the current release, and a hidden rawerhide that is the next release | 10:14 |
| f13 | one of hte issues I ran into this release was that if we were to let F11 be rawhide earlier, we have no where to redirect mirrormanager requests for fedora '10' | 10:14 |
| wwoods | making the final freeze a serious you-better-have-your-bits-in-damn-it Final Friggin' Freeze | 10:14 |
| f13 | we'd effectively cut off anything but pushed updates for our 10 users. | 10:14 |
| notting | i would think (confused users who want rawerhide) << (confused users who think what's there in Everything is final) | 10:16 |
| notting | unless we push it to test/ | 10:16 |
| wwoods | ah, so we solve that by populating the Everything repo before switching the fedora-release file | 10:16 |
| notting | maybe push to a RC/ tree under test | 10:16 |
| f13 | notting: could do that too | 10:16 |
| f13 | that would probably be less confusion | 10:17 |
| f13 | confusing. | 10:17 |
| f13 | We would have to time the composes differently too | 10:17 |
| f13 | decide which tree starts at 0600 UTC and which one starts either at 0000 UTC or 1200 UTC | 10:17 |
| f13 | It'll also fracture the testing community, which is a pretty important thing to consider | 10:19 |
| f13 | anyway, don't need to decide that now. | 10:21 |
| f13 | anything else on the release of Fedora 10? | 10:21 |
| notting | it's been awfully quiet on the mirror front (w.r.t, "can't sync", or "look, a leak!") | 10:22 |
| notting | f13: all the torrent stuff is ready for the flip? do we have pre-seeders? | 10:23 |
| wwoods | I think fracturing is not a huge worry, since a) moving to rawhide is opt-in, and b) early rawhide is widely known to be wildly unstable | 10:23 |
| wwoods | (or at least perceived as such) | 10:23 |
| f13 | notting: skvidal was taking care of the torrents, they should be good to go, and we have a few pre-seeders who were syncing content over the weekend, I'm going to ping them and make sure they got content. | 10:23 |
| f13 | notting: it has been quiet, but I think that's mostly because we haven't tried to do a 3 day full sync up like we usually do. We actually gave the mirrors enough time to sync without hysterics | 10:24 |
| notting | f13: #1093 can be closed? | 10:25 |
| f13 | .rel 1093 | 10:25 |
| zodbot | f13: #1093 (Prepare 0-day updates repos for F10 updates.) - Fedora Release Engineering - Trac - https://fedorahosted.org/rel-eng/ticket/1093 | 10:25 |
| f13 | ah yeah, that can be closed as of this morning by doing the mirrormanager change. | 10:25 |
| notting | ah. dependency errors in updates-testing. *sigh* | 10:26 |
| f13 | yeah, goal for F11 dev cycle is to finally address some of that. | 10:26 |
| lmacken | update IDs are still busted. I'm going to push out a fixed bodhi-server today, and reassign about 200 or so IDs. | 10:29 |
| f13 | nod | 10:29 |
| lmacken | I'll send out an errata with the fixed IDs as well | 10:29 |
| notting | oh, i noticed a bunch of updates@fp.o mail directly to me with this last push - was that intentional? | 10:29 |
| lmacken | no email settings were changed in bodhi | 10:30 |
| notting | *shrug* ok. f13: any non-f10 stuff? | 10:32 |
| * notting looks at .rel 24 | 10:32 | |
| f13 | .rel 24 | 10:32 |
| zodbot | f13: #24 (spins for Fedora 9) - Fedora Release Engineering - Trac - https://fedorahosted.org/rel-eng/ticket/24 | 10:32 |
| f13 | (zodbot is kind of dumb) | 10:32 |
| f13 | that's still jwb's item. I keep throwing him under the bus on this one | 10:32 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora releng - F11 | 10:32 | |
| f13 | so, first order of business for F11 is to nail down a schedule | 10:32 |
| f13 | we got an end point approved by FESCo, now we have to fill it out, and start discussing some intra-schedule changes | 10:33 |
| f13 | I had some ideas regarding feature freeze and wwoods has some thoughts regarding the tail end of the schedule | 10:33 |
| f13 | poelcat: are you around? | 10:33 |
| poelcat | yes | 10:33 |
| f13 | ok | 10:35 |
| f13 | poelcat: so you know my thoughts already, staggering beta and feature freeze | 10:35 |
| f13 | wwoods: have you talked to poelcat about your thoughts? | 10:36 |
| poelcat | f13: yes i formulated that here: http://poelstra.fedorapeople.org/schedules/f-11/f-11-standard.html | 10:36 |
| poelcat | spot also put forth a proposal captured in the other scenario (links at the top) | 10:36 |
| f13 | nod | 10:38 |
| wwoods | well, Rawhide (F12) branching isn't on there, dunno if that's something that usually goes on the schedule | 10:39 |
| wwoods | mostly my thoughts are messaging-related: the Final Freeze has been a Semifinal Slush for a couple releases now | 10:39 |
| wwoods | I want it to be clearer that all intended final builds *must* be ready by Preview | 10:40 |
| wwoods | and I believe f13 and I discussed actually renaming Preview to RC1 | 10:40 |
| poelcat | wwoods: yes, branching does go on there.. i was attempting to keep it as highlevel as possible right now | 10:40 |
| wwoods | poelcat: gotcha | 10:40 |
| wwoods | anyway - we called it Preview because we had a history of having a long slush there and calling it a Release Candidate would have been disingenuous | 10:41 |
| wwoods | one option was to call it something more accurate. the other option is to actually have that be the Final Freeze so that that build could actually be considered a Release Candidate | 10:42 |
| wwoods | we chose the easy way out - the name change | 10:42 |
| f13 | also, preview came one release because we had a /terrible/ test3 and wound up doing a test4, and people kept asking for that to carry forward | 10:42 |
| wwoods | that turns out to be hellish on testing | 10:42 |
| wwoods | I'm probably not the only one who's worn out from the last-minute "oh hell quick quick quick rebuild {anaconda,kernel,...}" panics | 10:42 |
| notting | so, the only difference from a milestone standpoint with spot's schedule is moving a week between alpha and beta? | 10:43 |
| wwoods | it's bad for quality, makes it way harder to get bits into the hands of testers, and hard on all of us personally | 10:43 |
| wwoods | the flipside is that it gives developers slightly less time to get in their final bits | 10:44 |
| wwoods | but then, development *always* expands to fill the time allotted | 10:44 |
| poelcat | notting: feature freeze is offset in the not-spot scenario | 10:44 |
| f13 | notting: pardon? | 10:44 |
| wwoods | so I don't think this plan hurts development unless we surprise people with the change | 10:44 |
| f13 | wwoods: what is the main problem you wish to solve with the proposed schedule change? | 10:45 |
| f13 | wwoods: lets look at it from that point of view | 10:45 |
| notting | poelcat: oh, blah, taskjuggler silliness again | 10:45 |
| wwoods | too much churn during the "final" freeze | 10:45 |
| f13 | What do we think leads to the churn? | 10:46 |
| f13 | discovery of bugs due to people using Preview? | 10:46 |
| notting | poelcat: the 'duration' numbers for the alpha/beta/preview have zero bearing on the actual duration of those cycles | 10:46 |
| f13 | putting off of fixing bugs until final freeze? | 10:46 |
| wwoods | well, the glibc final bits weren't delivered until.. what, a week ago? | 10:46 |
| f13 | wwoods: misnomer | 10:46 |
| f13 | wwoods: the actual bits were the same, they just got a soname change | 10:47 |
| f13 | which yeah, that blows and we probably shouldn't have allowed that change. | 10:47 |
| wwoods | okay, but that sort of packaging change should happen at the final freeze | 10:47 |
| wwoods | but, yes, that's a bad example. | 10:47 |
| f13 | the Preview release served a number of purposes | 10:47 |
| wwoods | we allow for a *lot* of churn in key packages, typically relating to features that aren't stable yet | 10:48 |
| f13 | it gave folks a last snapshot before we made the release to test with | 10:48 |
| f13 | it gave folks a snapshot of all the things we fixed since the last snapshot, or for many people, since Beta | 10:48 |
| * notting wonders how bad other PM software must be if we're actually using taskjuggler | 10:48 | |
| wwoods | right, I'm not suggesting we drop Preview | 10:48 |
| f13 | it also served as the "these are the frozen bits" snapshot | 10:48 |
| wwoods | but.. let's see. did we revert back to Pidgin post-Preview? | 10:49 |
| f13 | with lots of discussion, yes | 10:49 |
| wwoods | those sorts of decisions: - "are these bits the ones we want in Final | 10:49 |
| wwoods | " - should be settled *before* Preview | 10:49 |
| wwoods | we should *actually* have all the features testable in Beta, and anything that's not present should be deferred at that point | 10:50 |
| f13 | I agree, but are we going to be so inflexible so that we would absolutely refuse any change post preview, when it's obviously a good change? | 10:50 |
| wwoods | we should *actually* discuss the readiness of those features and decide which ones are going to be OK and which ones aren't *before* Preview | 10:50 |
| f13 | my thoughts were that freezes make it so that the only changes are changes we expect | 10:51 |
| wwoods | I'm not suggesting that, because there's always going to be stuff that needs a second thought after Preview | 10:51 |
| f13 | be that some change, little change, or no change | 10:51 |
| wwoods | it's unavoidable and we've accepted that truth | 10:51 |
| wwoods | I guess the main source of churn is new features | 10:52 |
| f13 | if we're not happy with the changes that are happening at freeze time, isn't it our place to refuse those changes from happening? | 10:52 |
| wwoods | Yes. So the simplest way to shape up Final Freeze would be to get more stringent about feature requirements at Beta and Preview | 10:53 |
| f13 | I guess, is the problem that we're taking the changes, or is the problem that the change was noticed as being needed and presented to us? | 10:53 |
| notting | so, one concern about the actual schedule, as presented - freezing *directly* after fudcon may not be useful | 10:53 |
| wwoods | the problem appears to be that sometimes we've committed to delivering stuff that might not be ready until well after the Final Freeze | 10:54 |
| f13 | notting: I had some concern over that as well, not for the changes considered during fudcon, but for the lack of bugfixing by key people during fudcon | 10:54 |
| wwoods | some stuff has no problem getting stable by Final Freeze - LIRC and webcam stuff comes to mind | 10:54 |
| f13 | wwoods: so would the solution be to be more strict about feature acceptance, backed up by a more strict schedule? | 10:55 |
| wwoods | but some things we accepted despite the fact that they probably weren't going to be 100% complete by Final Freeze. And I don't think that's a good idea. | 10:55 |
| wwoods | yeah. I don't like being more strict but I think we're leaning a little too far toward New Shiny at the cost of Stability | 10:56 |
| wwoods | which sucks because I *love* New Shiny | 10:56 |
| notting | f13: so, how would we adjust the schedule for this? where do we get that week back? | 10:57 |
| wwoods | but we release every 6 months - surely some things can wait an extra few months to be New Shiny *and* Stable | 10:57 |
| warren | Have we done a post-mortem to identify the actual "oops that was bad" bugs in F10 final? | 10:57 |
| notting | f13: given that there's two months between alpha and beta, I think we have plenty of room to push alpha bback | 10:57 |
| f13 | notting: "this" ? | 10:57 |
| notting | f13: fudcon | 10:57 |
| f13 | ah | 10:57 |
| warren | I'm only aware of two really bad bugs despite the churn at the end. | 10:57 |
| wwoods | part of the process is requiring more detailed feature scope/specifications by Alpha | 10:58 |
| wwoods | and using that to write a Test Plan by Beta | 10:58 |
| wwoods | which makes it much easier to decide whether to keep or defer a feature before Preview | 10:58 |
| f13 | notting: I think my off the cuff suggestion would be to move the alpha freeze one week later. | 10:58 |
| f13 | I think I talked with poelcat about this and he had some thoughts to the counter. | 10:58 |
| notting | why do we have a two-month alpha anyway? | 11:00 |
| warren | wwoods: what are actual bugs that are the result of the post-freeze churn? | 11:00 |
| wwoods | so, yes, more detailed Feature requirements (scope + test plan), and more strictness from FESCo/rel-eng regarding what we'll consider acceptable changes post-Final Freeze | 11:00 |
| f13 | notting: because we usually have a 2 month alpha? | 11:00 |
| spot | guys, i have to go to a 2 PM meeting | 11:01 |
| f13 | spot: ok, thanks. | 11:01 |
| f13 | wwoods: would it help if post-freeze tag requests had more input from QA? | 11:01 |
| f13 | I would really hate going down the route of dev ack, qa ack | 11:01 |
| wwoods | no, I think we have sufficient input there | 11:01 |
| wwoods | yeah I don't want that either | 11:01 |
| poelcat | f13: all I can remember re: fudcon was alpha is non-block freeze so it wasn't that critical? | 11:01 |
| wwoods | I think most of the changes need to made at the Feature Process level | 11:02 |
| f13 | poelcat: it's non-blocker, but it's still somewhat critical to get it out | 11:02 |
| f13 | wwoods: ok. Lucky for you both the feature person and the schedule person are the same. | 11:02 |
| wwoods | indeed! | 11:02 |
| f13 | poelcat: I think it's prudent to allow some post-fudcon time to fix up any bugs that got ignored during that time, so can we push that freeze date out a week, and then use that as the basis of a schedule for releng to vote on next week? | 11:04 |
| notting | f13: we do? | 11:04 |
| wwoods | I think the only real change needed at the rel-eng level is early (and loud) announcements that Final Freeze is supposed to be FINAL and we only accept builds that fix blocker/target bugs after that | 11:04 |
| f13 | notting: we do what? | 11:04 |
| poelcat | f13: of the two proposed schedules... which one is being selected to make those changes to? | 11:04 |
| notting | f13: two month alphas | 11:05 |
| f13 | poelcat: both would need that treatment wouldn't they? | 11:05 |
| notting | poelcat: it doesn't matter | 11:05 |
| f13 | poelcat: don't both schedules have alpha freeze at the same time? | 11:05 |
| f13 | notting: I was just going by the research that spot and poelcat had done in the past | 11:05 |
| poelcat | f13: alpha freeze is the same; feature freeze is not | 11:05 |
| notting | poelcat: ... we're talking about moving alpha. feature freeze is irrelevant, ergo it gets moved in both | 11:07 |
| f13 | poelcat: we're talking about alpha freeze, which has a conflict with fudcon | 11:07 |
| notting | f13: f10 was a shade under two months. f9 was 1 1/2 - 1 3/4. f8 was 1 month | 11:07 |
| f13 | notting: interesting. | 11:08 |
| f13 | would it make sense then | 11:08 |
| f13 | n/m | 11:08 |
| poelcat | f13: okay; move alpha freeze one week later? | 11:10 |
| f13 | please | 11:10 |
| * poelcat is also going to reduce down to ONE scenario to argue about :) | 11:10 | |
| f13 | heh | 11:10 |
| f13 | moving alpha freeze a week later should also move alpha release a week later | 11:11 |
| f13 | but that should be the end of that ripple effect | 11:11 |
| poelcat | correct. that is what I was going to do | 11:11 |
| poelcat | originally i was fearing more was being suggested :) | 11:11 |
| poelcat | one other question remains unanswered: three freezes up to beta | 11:12 |
| poelcat | 1) feature freeze | 11:12 |
| poelcat | 2) critical package freeze | 11:12 |
| poelcat | 3) beta freeze | 11:12 |
| poelcat | should they all be separated by a week? | 11:12 |
| notting | poelcat: is there a way to fix it so that the durations for '... release' consistently are from public availability to public availability of next milestone? | 11:12 |
| notting | i personally think #1 and #2 should be coincidental | 11:12 |
| f13 | poelcat: I think I made a mistake a bit with pushing for critical package freeze. That's something we'll want to approach each individual package and ask them for consideration | 11:13 |
| f13 | poelcat: so one week between feature freeze and beta freeze. | 11:13 |
| poelcat | f13: and remove 'critical package' freeze from proposed schedule? | 11:14 |
| f13 | yeah | 11:14 |
| poelcat | notting: i'm not sure how to tweak that... the reporting, as you've commented, leaves a little to be desired :) | 11:15 |
| poelcat | f13: okay I will make those changes and update the ticket | 11:15 |
| poelcat | wwoods: let me know what changes you'd like to make the feature process/policy, etc. | 11:16 |
| f13 | thanks, we should send mail / poke on IRC that we'll vote on it next week | 11:16 |
| wwoods | poelcat: I think it was discussed in the FESCo meeting last week but it will probably come up again. | 11:16 |
| wwoods | I've been trying to make sure the relevant sections in the empty Feature page are clearer, but let me know if I've missed anything | 11:17 |
| -!- f13 changed the topic of #fedora-meeting to: Fedora releng - open floor | 11:18 | |
| f13 | anything else before we wrap? | 11:18 |
| poelcat | wwoods: i need to know which feature pages you believe need more substance from a QA perspective... i'm doing a general review at approval time and thereafter just checking for status | 11:18 |
| wwoods | poelcat: right, I'll be on hand for FESCo meetings wherever possible to ensure that's the case | 11:19 |
| wwoods | (except probably not this week since I'll be in Alabama tomorrow through Saturday) | 11:20 |
| * mitr raises hand | 11:21 | |
| mitr | First code drop for the signing server is at http://people.redhat.com/mitr/rpmsigner-0.tar.bz2 . I'd very much appreciate if someone could take a look and check the interface is usable and the functionality sufficient. | 11:22 |
| mitr | I'd also like to temporarily get a higher-privileged account for koji (allowed to store signatures and to authenticate as other users) to test the signature storing side. Who can I calk to about that? | 11:23 |
| mitr | s/calk/talk/ | 11:23 |
| mbonnet | mitr: why do you need to auth as other users? | 11:24 |
| f13 | and could we actually do this with a test koji instance, rather than the production one? | 11:25 |
| mitr | mbonnet: The idea is that the user's "client" does not download the signed RPM locally in order to store the signature; instead a helper server that runs near other koji servers stores the signature on behalf of the user (who is authenticated using a SSL certificate) | 11:25 |
| mitr | f13: Is there a test instance available? If not, I can of course set one up, but i was hoping to avoid that work. | 11:26 |
| mbonnet | mitr: you might want to talk to mikem23, he has an internal test instance | 11:26 |
| mitr | mbonnet: Thank you. | 11:26 |
| f13 | anything else? | 11:28 |
| dgilmore | mitr: why do you have pygpgme-0.1 python-nss-0.1 in the tarball? rather than using system provided versions | 11:29 |
| mitr | dgilmore: They are slightly patched. The patches are now in bugzilla, but not accepted upstream yet. | 11:29 |
| mitr | (#472805, #470271, #470272) | 11:30 |
| f13 | ok, this conversation can continue, but I'm going to close the meeting, thanks all! | 11:30 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!