ReleaseEngineering/Meetings/2007-jun-11

Fedora Release Engineering Meeting :: Monday 11-Jun-2007

Buildroot contents for updates

• The buildroot used for fedora 7 updates building is not self updating. It only contains things from f7-gold, and stable released updates. This means that one update candidate cannot be built against another update candidate without rel-eng interaction. We haven't been very vocal about this yet, and it isn't documented anywhere.
• Do we want to adjust things or leave them be?
• See IRC log for discussion points

Decision: Talk to lmacken about modifying Bodhi to do a sanity check on published buildroot contents before allowing an update be pushed. After that we will make the dist-fc7-build buildroot auto-update with -candidate builds and see what happens.

Expand Standard Operating Procedures (SOPs)

• f13
• rel-eng has more and more people helping out so we want to make sure that things are well documented
• Infrastructure team started doing SOP/ pages (Standard Operating Procedure) and they are proving very useful. I'd like to start doing them for Release Engineering too
• thinking of ReleaseEngineering/SOP/<task> as the layout, the SOP page itself could be a list of pages and info on how to add an SOP
• Proposal: As a release engineer, as you do tasks for Fedora, check to see if there is an SOP/<task> page. If there isn't, create one and poke rel-eng folks for review.
• This is more of a mandate than a vote item.

Early Torrent Release - Rahul Sundaram

• explore possibility of doing early torrent releases.
• See IRC log for discussion of pros and cons and affect on mirrors

Decision: Investigate with Infrastructure team ways of getting more mirrors to participate in seeding the torrent (early?). Do not release torrent to general public before the agreed upon coordinated release date/time.

Upgrade path enforcement - Rahul Sundaram

• Discussion surrounding policy that upgrade path should not break from a particular point on, for example, after Test1 or Test2--enforced by Release Engineering.
• See IRC log for discussion details

Decision: Recommend to FESCo policy that upgrade path never be compromised either by removing builds or NEVR regressions. File RFE in Koji to enforce rule at build time

IRC Transcript

f13 changed the topic of #fedora-meeting to: RELENG-Meeting - Buildroot contents for updates - JesseKeating		<a href="#t12:57" class="time">12:57</a>
f13	Meeting time coming up.	<a href="#t12:57" class="time">12:57</a>
f13	Release Engineering meeting starting now.	<a href="#t12:59" class="time">12:59</a>
f13	jwb_gone: notting: jeremy: rdieter: warren: wwoods: ping	<a href="#t13:00" class="time">13:00</a>
notting	yes?	<a href="#t13:00" class="time">13:00</a>
rdieter	here	<a href="#t13:00" class="time">13:00</a>
* nirik sits in the bleachers with his coffee.		<a href="#t13:00" class="time">13:00</a>
f13	notting: just role call.	<a href="#t13:00" class="time">13:00</a>
wwoods	ptchu	<a href="#t13:00" class="time">13:00</a>
wwoods	curmudgeonly grammar teacher living in my frontal lobe requires me to say: it's roll call.	<a href="#t13:01" class="time">13:01</a>
* jeremy is here		<a href="#t13:01" class="time">13:01</a>
wwoods	like honor roll.	<a href="#t13:01" class="time">13:01</a>
notting	wwoods: cinnamon roll?	<a href="#t13:01" class="time">13:01</a>
notting	pork roll?	<a href="#t13:01" class="time">13:01</a>
jeremy	my clock is just slow :)	<a href="#t13:01" class="time">13:01</a>
f13	wwoods: but what if I'm asking to see who's here to fill what role?	<a href="#t13:01" class="time">13:01</a>
f13	role roll call?	<a href="#t13:02" class="time">13:02</a>
* mmcgrath pongs		<a href="#t13:02" class="time">13:02</a>
wwoods	f13: totally	<a href="#t13:02" class="time">13:02</a>
wwoods	and if you're playing back a filmstrip of this event, you can say: roll role roll call	<a href="#t13:02" class="time">13:02</a>
f13	ok, first topic is kind of fun.	<a href="#t13:02" class="time">13:02</a>
warren	fun	<a href="#t13:02" class="time">13:02</a>
f13	As it currently stands, the buildroot used for fedora 7 updates building is not self updating.  It only contains things from f7-gold, and stable released updates.	<a href="#t13:03" class="time">13:03</a>
f13	this means that one update candidate cannot be built against another update candidate without rel-eng interaction.	<a href="#t13:03" class="time">13:03</a>
f13	We haven't been very vocal about this yet, and it isn't documented anywhere.  I have to ask if we want to adjust things or leave them be.	<a href="#t13:03" class="time">13:03</a>
mmcgrath	f13: was this by design or an oversight?	<a href="#t13:04" class="time">13:04</a>
f13	mmcgrath: design	<a href="#t13:04" class="time">13:04</a>
wwoods	I think we want to adjust that. People have been complaining about how huge a PITA it is to get several interdependent packages built	<a href="#t13:04" class="time">13:04</a>
f13	it's how things work internally since before I got here.	<a href="#t13:04" class="time">13:04</a>
rdieter	might be worth trying, anyone against self-updated buildroots?	<a href="#t13:04" class="time">13:04</a>
* mmcgrath just wonders what we might break if we change it.		<a href="#t13:04" class="time">13:04</a>
wwoods	oh. then how did updated stuff get put into the buildroot for interdependent builds?	<a href="#t13:04" class="time">13:04</a>
notting	nothing would break until someone puts something broke in updates-candidate and someone builds against it	<a href="#t13:05" class="time">13:05</a>
f13	mmcgrath: we run the risk of broken buildroots, things built against things that don't go out as updates, etc...	<a href="#t13:05" class="time">13:05</a>
warren	Here's one possible solution, similar to the fedora-cvs work queue.  Create a fedora-rel-eng work queue, where people can ask for that kind of request in a controlled manner.  We can use that until we have a more automated process in packagedb.	<a href="#t13:05" class="time">13:05</a>
f13	wwoods: we have a 'dist-fc7-override' tag that we can tag builds with to make them available.	<a href="#t13:05" class="time">13:05</a>
nirik	how many requests has releng had to add something to the buildroots?	<a href="#t13:05" class="time">13:05</a>
jeremy	warren: you more want to eventually get to where people can specify "use these unreleased updates" for my build...  not something in the packagedb	<a href="#t13:05" class="time">13:05</a>
f13	I'm bringing pjones in, I know he's had some opinions on this in the past.	<a href="#t13:05" class="time">13:05</a>
wwoods	f13: right, which requires rel-eng intervention (and extra action by maintainers --> OMGWTFREGRESSION)	<a href="#t13:06" class="time">13:06</a>
warren	f13, oops, I shouldn't have added a package to dist-fc7-build I'm guessing.  I'll have to undo that.	<a href="#t13:06" class="time">13:06</a>
pjones	hey, so where are we?	<a href="#t13:06" class="time">13:06</a>
wwoods	how did this work for Core / Extras?	<a href="#t13:06" class="time">13:06</a>
rdieter	wwoods: Extras self-updated, Core, dunno (not?)	<a href="#t13:06" class="time">13:06</a>
nirik	extras always got everything after builing.	<a href="#t13:06" class="time">13:06</a>
notting	wwoods: extras always built against latest, even unreleased	<a href="#t13:06" class="time">13:06</a>
notting	wwoods: core required intervention	<a href="#t13:06" class="time">13:06</a>
f13	pjones: basically I've stated the current situation (not updating) and gathering thoughts/opinions on either keeping it like it is, or changing it and risking the breakage.	<a href="#t13:06" class="time">13:06</a>
f13	mschwendnt (hwoever the hell you spell that) and dgilmore had some stats for how often the buildroot got fubar in Extras	<a href="#t13:07" class="time">13:07</a>
nirik	I know it will be fun for me on the next Xfce update... there are lots of dependency chains there... 3 inital packages, then everythign needs them and each one in the row after that...	<a href="#t13:07" class="time">13:07</a>
wwoods	It would seem like we need some kind of a hybrid where people can create private branched buildroots for building their stuff, or something?	<a href="#t13:07" class="time">13:07</a>
pjones	f13: at the very least, if we leave the default like it is, we need the ability to make it pull from updates and such via BuildRequires:	<a href="#t13:07" class="time">13:07</a>
f13	pjones: well, the buildroot currently is populated with f7-gold and stable updates	<a href="#t13:07" class="time">13:07</a>
wwoods	I admit not having deep knowledge of how much overhead that would require. All I know is that a bunch of maintainers are unhappy about it. Which is not news to anyone.	<a href="#t13:08" class="time">13:08</a>
f13	pjones: it just doesn't have any updates-testing or -candidate builds.	<a href="#t13:08" class="time">13:08</a>
pjones	so if, for example, I know a new anaconda needs a newer kudzu which I've just built, then I can BR: it and have that work.	<a href="#t13:08" class="time">13:08</a>
nirik	how about a bodhi tag/box? "add to buildroot for more builds"?	<a href="#t13:08" class="time">13:08</a>
mdomsch	pjones, BR with version?	<a href="#t13:08" class="time">13:08</a>
f13	pjones: in the current situation, unless kudzu has been pushed out as stable, you'd have to request rel-eng interaction to tag the kudzu build.	<a href="#t13:08" class="time">13:08</a>
pjones	mdomsch: yes	<a href="#t13:09" class="time">13:09</a>
mdomsch	pjones, even then, you need to have the repo available for mock to use it	<a href="#t13:09" class="time">13:09</a>
pjones	mdomsch: agreed	<a href="#t13:09" class="time">13:09</a>
f13	mdomsch: if the buildroot was self updating, chain-build would work	<a href="#t13:09" class="time">13:09</a>
f13	almost.	<a href="#t13:09" class="time">13:09</a>
notting	f13: that's a big win for peopel	<a href="#t13:09" class="time">13:09</a>
rdieter	I'm of the KISS mind, auto-update buildroot, rel-eng can step in and fix any breakage (ie, untag bustedness) if need be.	<a href="#t13:09" class="time">13:09</a>
pjones	f13: chain-build has other problems though.  not the least of which being the syntax is weird	<a href="#t13:09" class="time">13:09</a>
f13	I don't think chainbuild has the concept of 'this was already built, but wait unti it's available to build me'	<a href="#t13:09" class="time">13:09</a>
f13	pjones: yeah, syntax is interesting.  NOt sure how to make it better.  Open to suggestions	<a href="#t13:10" class="time">13:10</a>
rdieter	f13: chainbuild worked for me that way, when I tried it.	<a href="#t13:10" class="time">13:10</a>
pjones	rdieter: *nod*.  We should be able to tell "oh, kudzu is fucked, we need to roll it out as well as ${dependent packages that got built}"	<a href="#t13:10" class="time">13:10</a>
f13	rdieter: oh cool, maybe it is smart enough (:	<a href="#t13:10" class="time">13:10</a>
f13	pjones: my worry is that $(dependent packages that got buitl) may have made it out to stable updates	<a href="#t13:10" class="time">13:10</a>
f13	and then it's not so easy/forgiving to just make it go away	<a href="#t13:11" class="time">13:11</a>
pjones	rdieter: I think we have problems where an update isn't available to build with more often than we have problems where an update was bad.	<a href="#t13:11" class="time">13:11</a>
pjones	f13: that seems like something we should be able to preclude from happening	<a href="#t13:11" class="time">13:11</a>
rdieter	pjones: agreed.	<a href="#t13:11" class="time">13:11</a>
f13	pjones: with some work sure, bodhi could query to see if any component in the buildroot for an update is as of yet unreleased.	<a href="#t13:11" class="time">13:11</a>
pjones	f13: nod.	<a href="#t13:11" class="time">13:11</a>
warren	f13, good idea.	<a href="#t13:11" class="time">13:11</a>
* f13 signs lmacken up for more work.		<a href="#t13:12" class="time">13:12</a>
rdieter	A compromise maybe would be to enable updates-testing in the buildroot?	<a href="#t13:12" class="time">13:12</a>
f13	rdieter: that's not good enough I don't thik.	<a href="#t13:12" class="time">13:12</a>
f13	think.	<a href="#t13:12" class="time">13:12</a>
jeremy	f13: with that, I'd be okay with allowing updates-candidates in buildroots.  but it's going to cause problems	<a href="#t13:12" class="time">13:12</a>
f13	rdieter: you may want to issue a big stack as one update.	<a href="#t13:12" class="time">13:12</a>
jeremy	eg, jakub builds new glibc, it's in updates-testing and he wants it to be there a week	<a href="#t13:12" class="time">13:12</a>
rdieter	f13: ok, nevermind.	<a href="#t13:12" class="time">13:12</a>
jeremy	people that want to build and immediately push (or security updates, etc) will get flagged	<a href="#t13:12" class="time">13:12</a>
f13	jeremy: well, jakub builds into dist-fc7-candidate anyway, something completely not imported for his own personal tests, then tags it for updates-cnadidate by hand when he knows it's good	<a href="#t13:13" class="time">13:13</a>
f13	but yes, I can see that causing problems, so perhaps an override option.	<a href="#t13:13" class="time">13:13</a>
f13	unless that will break deps :/	<a href="#t13:13" class="time">13:13</a>
f13	So I'm willing to experiment with Fedora 7 updates to see how it goes, once we get some bodhi love.	<a href="#t13:14" class="time">13:14</a>
warren	f13, how could bodhi know if something that was in the buildroot is not yet pushed?  Does mock output list *all* packages that were installed?	<a href="#t13:14" class="time">13:14</a>
pjones	f13: perhaps allow package builders to specifically tag their updates-testing things for building?  you'd probably still need the "this can't be released until that is" infrastructure, but it wouldn't have the problem jeremy mentioned	<a href="#t13:14" class="time">13:14</a>
f13	warren: koji tracks buildroot components for a build.	<a href="#t13:15" class="time">13:15</a>
warren	f13, ah.	<a href="#t13:15" class="time">13:15</a>
wwoods	I thought we were going to start allowing updates to have a set of packages?	<a href="#t13:16" class="time">13:16</a>
nirik	yeah, I think it would make sense to leave default as it is now, and allow a checkbox for maintainers in bodhi to add their update-testing update to the buildroot?	<a href="#t13:16" class="time">13:16</a>
f13	pjones: that's a possiblity.  Would have to think about how to construct that workflow	<a href="#t13:16" class="time">13:16</a>
notting	nirik: that really doesn't help with things like chain-build	<a href="#t13:16" class="time">13:16</a>
warren	nirik, oh, so a third unpublished layer?	<a href="#t13:16" class="time">13:16</a>
f13	ah right.	<a href="#t13:16" class="time">13:16</a>
f13	that doesn't help chain-build.	<a href="#t13:16" class="time">13:16</a>
nirik	true.	<a href="#t13:16" class="time">13:16</a>
f13	so...	<a href="#t13:16" class="time">13:16</a>
warren	bodhi could push to THREE layers instead of the current two.	<a href="#t13:16" class="time">13:16</a>
warren	unpushed -> buildrootonly -> updates-testing -> updates	<a href="#t13:17" class="time">13:17</a>
nirik	well, it already has pending/updates-testing/updates	<a href="#t13:17" class="time">13:17</a>
f13	Proposal:  We talk to lmacken about modifying bodhi to do a sanity check on published buildroot contents before allowing an update be pushed.  We then make the buildroot self updating and see what happens.	<a href="#t13:17" class="time">13:17</a>
warren	ah	<a href="#t13:17" class="time">13:17</a>
rdieter	f13: +1	<a href="#t13:17" class="time">13:17</a>
f13	warren: you'd still introduce a break in the workflow where you can't rely upon chain-build.	<a href="#t13:17" class="time">13:17</a>
warren	nirik, what does pending mean?	<a href="#t13:17" class="time">13:17</a>
warren	f13, +1	<a href="#t13:17" class="time">13:17</a>
f13	warren: it means it hasn't gone out in updates-testing yet	<a href="#t13:17" class="time">13:17</a>
nirik	warren: someone added that they want it as an update, but hasn't pushed it to updates-testing yet	<a href="#t13:18" class="time">13:18</a>
wwoods	f13: update pushed live, or pushed to -testing?	<a href="#t13:18" class="time">13:18</a>
pjones	f13: as an aside, the problem with chain-build is that it doesn't really act on a checked out tree	<a href="#t13:19" class="time">13:19</a>
pjones	f13: +1 on your proposal above, btw.	<a href="#t13:19" class="time">13:19</a>
f13	pjones: lets talk about chain-build over in #fedora-devel after this with mbonnet ok?	<a href="#t13:20" class="time">13:20</a>
pjones	ok	<a href="#t13:20" class="time">13:20</a>
f13	wwoods: I don't understand the question?	<a href="#t13:20" class="time">13:20</a>
abadger1999	f13: +1	<a href="#t13:20" class="time">13:20</a>
f13	wwoods: oh I see.  Well, it shouldn't allow something to go into -testing unless what was used to produce it is on the way to -testing, ditto final.	<a href="#t13:21" class="time">13:21</a>
wwoods	f13: I'm trying to clarify "pushed" in your proposal - pushed where?	<a href="#t13:21" class="time">13:21</a>
f13	although we could be more relaxed with -testing.	<a href="#t13:21" class="time">13:21</a>
wwoods	ah, gotcha	<a href="#t13:21" class="time">13:21</a>
wwoods	+1 either way, but I wanted to be clear	<a href="#t13:21" class="time">13:21</a>
f13	k.  I'll work with lmacken to see which is more feasable.	<a href="#t13:22" class="time">13:22</a>
f13	This may take a while to implement so we'll still need documented workflow for people to ask rel-eng to tag things for them.	<a href="#t13:22" class="time">13:22</a>
f13	jeremy: notting: your votes?	<a href="#t13:23" class="time">13:23</a>
notting	f13: open the floodgates!	<a href="#t13:23" class="time">13:23</a>
jeremy	let's see what we can do...  I'm open to trying things, though still a little concerned for the obvious reasons	<a href="#t13:23" class="time">13:23</a>
f13	jeremy: ditto (:	<a href="#t13:23" class="time">13:23</a>
f13	Ok, so that passed.	<a href="#t13:23" class="time">13:23</a>
f13	poelcat: Decision: We talk to lmacken about modifying bodhi to do a sanity check on published buildroot contents before allowing an update be pushed.  We then make the dist-fc7-build buildroot auto-update with -candidate builds and see what happens.	<a href="#t13:24" class="time">13:24</a>
f13 changed the topic of #fedora-meeting to: RELENG-Meeting - Flesh out SOPs for rel-eng tasks - JesseKeating		<a href="#t13:24" class="time">13:24</a>
f13	so rel-eng has more and more people doing things for us, we'll want to make sure that things are well documented.	<a href="#t13:25" class="time">13:25</a>
f13	The Infrastructure team started doing SOP/ pages (Standard Operating Proceedure) and they are proving very useful.  I'd like to start doing them for Release Engineering too	<a href="#t13:25" class="time">13:25</a>
* mclasen mumbles about silly acronyms		<a href="#t13:26" class="time">13:26</a>
f13	I'm thinking of ReleaseEngineering/SOP/<task>  as the layout, the SOP page itself could be a list of pages and info on how to add an SOP	<a href="#t13:26" class="time">13:26</a>
f13	mclasen: suggestions?	<a href="#t13:26" class="time">13:26</a>
warren	f13, sounds good to me.	<a href="#t13:26" class="time">13:26</a>
mclasen	since you spell out release engineering there, why not spell out procedure as well	<a href="#t13:27" class="time">13:27</a>
f13	*shrug* I'm lazy, and there is already precidence for SOP/ in the Infrastructure pages	<a href="#t13:27" class="time">13:27</a>
* mclasen relurks		<a href="#t13:27" class="time">13:27</a>
f13	I don't really think it matters if the page is linked to from ReleaseEngineering.	<a href="#t13:27" class="time">13:27</a>
f13	Proposal: As a release engineer, as you do tasks for Fedora, check to see if there is an SOP/<task> page.  If there isn't, create one and poke rel-eng folks for review.	<a href="#t13:28" class="time">13:28</a>
f13	I guess this is more of a mandate than a vote item.	<a href="#t13:28" class="time">13:28</a>
f13	Ok, moving on	<a href="#t13:29" class="time">13:29</a>
f13 changed the topic of #fedora-meeting to: RELENG-Meeting - Early Torrent Release - RahulSundaram		<a href="#t13:29" class="time">13:29</a>
f13	mether wanted us to talk about doing early torrent releases.	<a href="#t13:29" class="time">13:29</a>
mmcgrath	f13: I found out at linux tag that OpenSuSE actually has some of their mirrors setup as torrent seeds.  We could also do that.	<a href="#t13:30" class="time">13:30</a>
f13	I'm personally not a huge fan of this, last time the mirror admins caught wind of something like this they got kinda pissy.	<a href="#t13:30" class="time">13:30</a>
wwoods	like.. torrents open up before the mirrors finish synching?	<a href="#t13:30" class="time">13:30</a>
jeremy	mmcgrath: some of our mirrors act as torrent seeds	<a href="#t13:30" class="time">13:30</a>
jeremy	mmcgrath: eg, I know kernel.org does	<a href="#t13:30" class="time">13:30</a>
* mmcgrath didn't know that.		<a href="#t13:30" class="time">13:30</a>
f13	wwoods: yeah, I don't recall from the email how soon he wanted it open.	<a href="#t13:30" class="time">13:30</a>
jeremy	f13: I'm not a fan, so -1 from me	<a href="#t13:30" class="time">13:30</a>
mmcgrath	how do we get the seed to them?	<a href="#t13:30" class="time">13:30</a>
f13	mmcgrath: it's done after release though, not before release.	<a href="#t13:30" class="time">13:30</a>
mmcgrath	ahh	<a href="#t13:30" class="time">13:30</a>
f13	mmcgrath: they probably just copy the mirrored isos into place then fire up bittorrent	<a href="#t13:31" class="time">13:31</a>
wwoods	I think I kind of fail to see the point	<a href="#t13:31" class="time">13:31</a>
jeremy	yeah, believe so	<a href="#t13:31" class="time">13:31</a>
* f13 tries to find the posting		<a href="#t13:31" class="time">13:31</a>
warren	A mirror with lots of bandwidth doesn't need the data to be useful to a torrent cloud.	<a href="#t13:31" class="time">13:31</a>
mmcgrath	wwoods: I think the point is to encourage torrent usage.	<a href="#t13:31" class="time">13:31</a>
warren	They will rapidly obtain it and become a seed within 10-20 minutes.	<a href="#t13:31" class="time">13:31</a>
mmcgrath	did anyone try the torrent right after release this time around?  I wonder what the speeds were.	<a href="#t13:31" class="time">13:31</a>
notting	f13: right, we don't actually publish the layout for torrent, though	<a href="#t13:31" class="time">13:31</a>
f13	Torrent only release	<a href="#t13:32" class="time">13:32</a>
f13	--------------------	<a href="#t13:32" class="time">13:32</a>
f13	I remember this was discussed before but when a new release is made and	<a href="#t13:32" class="time">13:32</a>
f13	we are waiting for the mirrors to sync why not do a torrent only release	<a href="#t13:32" class="time">13:32</a>
f13	immediately?	<a href="#t13:32" class="time">13:32</a>
rdieter	mmcgrath: I did, speed was good (not great).	<a href="#t13:32" class="time">13:32</a>
f13	The advantage is that we won't suffer as much from infrastructure and	<a href="#t13:32" class="time">13:32</a>
f13	mirror problems with the mad rush in everybody trying to get it at the	<a href="#t13:32" class="time">13:32</a>
f13	same time. Not sure if this has any disadvantages.	<a href="#t13:32" class="time">13:32</a>
mmcgrath	rdieter: but you didn't think "gosh this is so terrible I'll just go to download.fedora.redhat.com and get it from there" ? :)	<a href="#t13:32" class="time">13:32</a>
poelcat	mmcgrath: speed was not that great for at least 24 hours, maybe longer	<a href="#t13:32" class="time">13:32</a>
warren	why would  it be terrible?	<a href="#t13:32" class="time">13:32</a>
warren	The more people that download a torrent, the faster it goes.  A mirror server's bandwidth only accelerates it.	<a href="#t13:32" class="time">13:32</a>
rdieter	mmcgrath: nope, I think it took ~2 hrs for me.	<a href="#t13:33" class="time">13:33</a>
f13	Well, bittorrent is pretty damn dumb in how it works	<a href="#t13:33" class="time">13:33</a>
poelcat	warren: but not if everyone is starting at zero to begin with	<a href="#t13:33" class="time">13:33</a>
mmcgrath	warren: not if no one has it and everyone is hitting torrent.fp.o to get it in the first many hours.	<a href="#t13:33" class="time">13:33</a>
f13	so if there are a lot of seeders in the other side of hte world from you, and a couple local, torrent will try all of them anyway and you'll be stuck getting bits slowly	<a href="#t13:33" class="time">13:33</a>
notting	f13: so, disadvantages: 1) annoy mirrors 2) at most, you're getting a day or two extra release	<a href="#t13:33" class="time">13:33</a>
f13	notting: that was my thought too.	<a href="#t13:34" class="time">13:34</a>
mmcgrath	really we should ask ourselves "Is increasting torrent speed over the first 24 hours something we need to fix"	<a href="#t13:34" class="time">13:34</a>
f13	plus confusion in the masses as to what the release date really is	<a href="#t13:34" class="time">13:34</a>
f13	mmcgrath: We should always consider making things faster.	<a href="#t13:34" class="time">13:34</a>
mmcgrath	but at what cost!  Oh the humanity!	<a href="#t13:34" class="time">13:34</a>
f13	I think we could publish torrent layouts early or ask for mirrors officailly to help torrent.	<a href="#t13:34" class="time">13:34</a>
f13	so I'm interested in looking at improving that.  I don't thin kjust releasing the torrent to the general public is the way to achieve this though.	<a href="#t13:35" class="time">13:35</a>
wwoods	I think getting some mirrors to seed the torrents might be the smarter thing to do	<a href="#t13:35" class="time">13:35</a>
mmcgrath	<nod>	<a href="#t13:35" class="time">13:35</a>
wwoods	I don't think breaking the release dates is a good idea	<a href="#t13:35" class="time">13:35</a>
mmcgrath	we could limit early release to people with a fedoraproject.org account and ask them to start seeding early.	<a href="#t13:35" class="time">13:35</a>
warren	mmcgrath, how do you limit that?	<a href="#t13:35" class="time">13:35</a>
f13	Proposal:  Investigate with Infrastructure team ways of getting more mirrors to participate in seeding the torrent (early?).  Do not release torrent to general public before the agreed upon coordinated release date/time.	<a href="#t13:36" class="time">13:36</a>
warren	mmcgrath, torrent has no login	<a href="#t13:36" class="time">13:36</a>
poelcat	how much impact would there be if 20-30 individ people had adv access to GA IOS and started their own seed at the minute release went live? not before	<a href="#t13:36" class="time">13:36</a>
mmcgrath	warren: by restricting access to the .torrent file	<a href="#t13:36" class="time">13:36</a>
warren	mmcgrath, so it could be leaked easily	<a href="#t13:36" class="time">13:36</a>
mmcgrath	warren: your right, lets do nothing.	<a href="#t13:37" class="time">13:37</a>
warren	that'll work though	<a href="#t13:37" class="time">13:37</a>
notting	poelcat: such as.. mirrors?	<a href="#t13:37" class="time">13:37</a>
warren	mmcgrath, I'm not suggesting that we do nothing.	<a href="#t13:37" class="time">13:37</a>
nirik	with f7 it was leaked and appeared on some of the torrent sites... torrentspy, etc.	<a href="#t13:37" class="time">13:37</a>
poelcat	notting: no idividual people like you and me... running from home	<a href="#t13:37" class="time">13:37</a>
nirik	(before release)	<a href="#t13:37" class="time">13:37</a>
rdieter	f13: +1	<a href="#t13:37" class="time">13:37</a>
warren	Leaks are OK IMHO, but we better make sure the ACTUAL release is leaked.	<a href="#t13:38" class="time">13:38</a>
warren	(People don't receive a trojaned copy, or not quite final.)	<a href="#t13:38" class="time">13:38</a>
notting	i think mirrors would do better	<a href="#t13:38" class="time">13:38</a>
f13	I do too	<a href="#t13:38" class="time">13:38</a>
mmcgrath	f13: sounds good to me.	<a href="#t13:38" class="time">13:38</a>
f13	mirrors can make use of symlinks/hardlinks whatever to participate, mirrormanager could potentially help	<a href="#t13:38" class="time">13:38</a>
warren	When I ran a mirror with DS-3 bandwidth (45mbit) and I joined the user swarm, I was downloading FC1 ISO's at like 7MB/sec.	<a href="#t13:39" class="time">13:39</a>
warren	hmm... that doesn't add up.  we might have had an OC-3 by then.	<a href="#t13:39" class="time">13:39</a>
rdieter	afk 15 mins...	<a href="#t13:40" class="time">13:40</a>
* f13 pushes for more votes.		<a href="#t13:40" class="time">13:40</a>
warren	f13, +1	<a href="#t13:41" class="time">13:41</a>
f13	warren, notting, jeremy, wwoods ?	<a href="#t13:41" class="time">13:41</a>
jeremy	sure!	<a href="#t13:41" class="time">13:41</a>
wwoods	+1	<a href="#t13:41" class="time">13:41</a>
f13	poelcat: Decision: Investigate with Infrastructure team ways of getting more mirrors to participate in seeding the torrent (early?).  Do not release torrent to general public before the agreed upon coordinated release date/time.	<a href="#t13:42" class="time">13:42</a>
f13 changed the topic of #fedora-meeting to: RELENG-Meeting - Upgrade path enforcement - RahulSundaram		<a href="#t13:42" class="time">13:42</a>
f13	Another mether topic	<a href="#t13:42" class="time">13:42</a>
f13	Supported live upgrade from last test to general release	<a href="#t13:42" class="time">13:42</a>
f13	---------------------------------------------------------	<a href="#t13:42" class="time">13:42</a>
f13	This was agreed upon for Fedora 7 test 4 in QA meetings but we didn't	<a href="#t13:42" class="time">13:42</a>
f13	actually announce it. It would encourage us to be more careful in	<a href="#t13:42" class="time">13:42</a>
f13	pushing out any updates after the last test release and it would	<a href="#t13:42" class="time">13:42</a>
f13	encourage more people to try it out the development branch and provide	<a href="#t13:42" class="time">13:42</a>
f13	us feedback.	<a href="#t13:42" class="time">13:42</a>
f13	I think all he's looking for here is that we set policy that upgrade path shouldn't break from that point on, and we enforce that.	<a href="#t13:43" class="time">13:43</a>
warren	This means don't pull packages from rawhide in "oops" cases?	<a href="#t13:43" class="time">13:43</a>
f13	THere has been some discussion around making it so that upgrade path works from day one of a release cycle though, pre-test1 even.	<a href="#t13:43" class="time">13:43</a>
f13	warren: or revert with epoch	<a href="#t13:43" class="time">13:43</a>
wwoods	at the very least, we're saying that official support for (Last Test Release) -> (Final Release) upgrades should be considered	<a href="#t13:44" class="time">13:44</a>
warren	I'm all for disallowing pulling of packages from rawhide *always*.	<a href="#t13:44" class="time">13:44</a>
f13	I'm somewhat lazy and and would like to set policy only from say test2 or test3 on.	<a href="#t13:44" class="time">13:44</a>
warren	If you need to undo a mistake, use epoch.   People are really overly emotional about epoch.	<a href="#t13:44" class="time">13:44</a>
f13	but I can be convinced otherwise.	<a href="#t13:44" class="time">13:44</a>
wwoods	in theory there would only be critical fixes in that time period anyway	<a href="#t13:44" class="time">13:44</a>
f13	wwoods: "this time" we'll actually be good about that!	<a href="#t13:44" class="time">13:44</a>
warren	We lose testers whenever we require users to undo a mistake manually.	<a href="#t13:44" class="time">13:44</a>
f13	</straight_face>	<a href="#t13:44" class="time">13:44</a>
wwoods	right, we're saying test3 (where test3 is final) - I'm sure mether would be happy to have it at test2 or earlier	<a href="#t13:44" class="time">13:44</a>
notting	f13: an addict will never change until *they* decide to change	<a href="#t13:45" class="time">13:45</a>
wwoods	I'll leave it to you all to determine whether that's feasible or not	<a href="#t13:45" class="time">13:45</a>
wwoods	test2 would be even better because in theory that's the feature freeze	<a href="#t13:45" class="time">13:45</a>
warren	I would be OK with a compromise of test2.	<a href="#t13:45" class="time">13:45</a>
wwoods	so guaranteeing that users will be able to easily upgrade from test2 to final	<a href="#t13:45" class="time">13:45</a>
wwoods	will mean getting lots more testers by test2. or so we'd hope.	<a href="#t13:45" class="time">13:45</a>
f13	wwoods: that's not what we're saying.	<a href="#t13:45" class="time">13:45</a>
warren	f13, part of this is a rule, "After test2, rawhide yum update should not require manual intervention."?	<a href="#t13:46" class="time">13:46</a>
f13	wwoods: all I'm talking about right now is enforceing upgrade paths.	<a href="#t13:46" class="time">13:46</a>
f13	warren: I don't know if we can really ensure that either	<a href="#t13:46" class="time">13:46</a>
notting	f13: how is that different? why wouldn't upgrades work?	<a href="#t13:46" class="time">13:46</a>
* lmacken strolls in		<a href="#t13:46" class="time">13:46</a>
f13	notting: %post fuckups	<a href="#t13:46" class="time">13:46</a>
wwoods	yeah, why wouldn't that work? assuming that you've already installed test2 by a supported method..	<a href="#t13:46" class="time">13:46</a>
f13	other such things.	<a href="#t13:46" class="time">13:46</a>
f13	beyond the "fixing" of here's a new package	<a href="#t13:46" class="time">13:46</a>
f13	I don't want to put us on the line for some things that may be out of our control.  I'm happy to enforce upgrade paths, which goes a long long way to this, but I'm not going to say we can prevent manual intervention completely	<a href="#t13:47" class="time">13:47</a>
warren	f13, i'm unclear on what exactly this proposal would mean.	<a href="#t13:47" class="time">13:47</a>
f13	warren: from say test2 on, any new package build has to have a clean upgrade path from the previous released package	<a href="#t13:48" class="time">13:48</a>
warren	f13, will we stop the insanity of undoing a %{version} upgrade by removing it from rawhide and expecting users to manually downgrade?	<a href="#t13:48" class="time">13:48</a>
f13	no making builds dissappear, no downgrades without epoch	<a href="#t13:48" class="time">13:48</a>
abadger1999	Right. things like broken deps could also need manual intervention.	<a href="#t13:48" class="time">13:48</a>
warren	f13, is that the entire scope of this proposal?	<a href="#t13:48" class="time">13:48</a>
f13	warren: are you looking for something else?	<a href="#t13:49" class="time">13:49</a>
warren	no	<a href="#t13:49" class="time">13:49</a>
warren	I'd be happy with that.	<a href="#t13:49" class="time">13:49</a>
warren	+1 for after test2	<a href="#t13:49" class="time">13:49</a>
f13	Can we come up with a reasonable cutoff point?  forever?  test1/2/3?	<a href="#t13:49" class="time">13:49</a>
warren	I would prefer forever, but test2 is a good compromise.	<a href="#t13:49" class="time">13:49</a>
warren	or even test1	<a href="#t13:49" class="time">13:49</a>
wwoods	test2 seems like a reasonable compromise for the first run for this	<a href="#t13:50" class="time">13:50</a>
wwoods	test1 is a bit early - it's pre-feature freeze so we can't guarantee there won't be big changes between test1 and test2	<a href="#t13:50" class="time">13:50</a>
f13	perhaps rel-eng can suggest test2 to FESCo, and they'll ratify/adjust as necessary.	<a href="#t13:50" class="time">13:50</a>
f13	wwoods: good point, like packages going away	<a href="#t13:50" class="time">13:50</a>
warren	Who opposes "forever"?	<a href="#t13:50" class="time">13:50</a>
wwoods	since the goal of this change is to make upgrades from test->final much easier, we can't necessarily hold those promises for anything pre-feature-freeze	<a href="#t13:51" class="time">13:51</a>
f13	warren: I do.	<a href="#t13:51" class="time">13:51</a>
warren	f13, do you similarly oppose test1?	<a href="#t13:51" class="time">13:51</a>
f13	yeah, pretty much the same reasons.	<a href="#t13:51" class="time">13:51</a>
f13	up to test2 things are going to be somewhat fast and loose.	<a href="#t13:51" class="time">13:51</a>
warren	but you're OK with the test2 compromise?	<a href="#t13:51" class="time">13:51</a>
f13	yeah, I'm fine with test2 on	<a href="#t13:52" class="time">13:52</a>
warren	OK, let's do it.	<a href="#t13:52" class="time">13:52</a>
* f13 is interested in notting and jeremy's opinion.		<a href="#t13:52" class="time">13:52</a>
warren	jeremy said "sure!"	<a href="#t13:53" class="time">13:53</a>
notting	i'd sort of like earlier, but there are occasionally going to be things that have to 'go away'	<a href="#t13:53" class="time">13:53</a>
jeremy	as a general thing, it's fine... but as notting says, there are always going to be exceptions	<a href="#t13:53" class="time">13:53</a>
mclasen	of course, it doesn't mean that we shouldn't strive to have it at all times	<a href="#t13:54" class="time">13:54</a>
notting	exactly	<a href="#t13:54" class="time">13:54</a>
notting	we need to make things easier for users, not harder	<a href="#t13:55" class="time">13:55</a>
jeremy	yep, I'm 110% agreed.  just want to make sure we're realistic about the fact that we're not going to hit 100%	<a href="#t13:55" class="time">13:55</a>
f13	well, if that	<a href="#t13:55" class="time">13:55</a>
* warren tries to understand the logic of 110% not reaching 100%.		<a href="#t13:56" class="time">13:56</a>
f13	's the desire, should we just set policy to have upgrade paths from day one, and only make exceptions when necessary?	<a href="#t13:56" class="time">13:56</a>
jeremy	f13: yep	<a href="#t13:56" class="time">13:56</a>
warren	f13, +1	<a href="#t13:57" class="time">13:57</a>
warren	f13, does this mean builds in F7 should fail if it isn't already superceded by a build in devel now?	<a href="#t13:57" class="time">13:57</a>
jwb_gone	what do you do with packages that don't make it	<a href="#t13:58" class="time">13:58</a>
f13	jwb_gone: untag them!	<a href="#t13:58" class="time">13:58</a>
f13	oh wait...	<a href="#t13:58" class="time">13:58</a>
warren	jwb_gone, concrete example?	<a href="#t13:58" class="time">13:58</a>
jwb_gone	how does that fix the repo?	<a href="#t13:58" class="time">13:58</a>
notting	warren: huh?	<a href="#t13:58" class="time">13:58</a>
jwb_gone	foo-1.1.2.fc6 > foo-1.1.1.fc7	<a href="#t13:58" class="time">13:58</a>
jwb_gone	what do you do in that case?	<a href="#t13:58" class="time">13:58</a>
f13	we're talking about devel, not f7	<a href="#t13:58" class="time">13:58</a>
f13	warren: ^^	<a href="#t13:59" class="time">13:59</a>
warren	jwb_gone, epoch	<a href="#t13:59" class="time">13:59</a>
jwb_gone	was an example.  s/fc6/fc7 s/fc7/fc8	<a href="#t13:59" class="time">13:59</a>
warren	People really need to get over their fear of epoch.  It is better to use epoch than to break automated upgrades and to lose testing.	<a href="#t13:59" class="time">13:59</a>
jwb_gone	mether's option was to pull that package from the repo in that case.  i think that is wrong	<a href="#t13:59" class="time">13:59</a>
warren	Yes, mether is wrong in that case.	<a href="#t13:59" class="time">13:59</a>
f13	who does the epoch and build though?  Should we just file bugs and expect people to fix it, or will rel-eng have to go on a warpath at some point and fix all these things ourselves?	<a href="#t14:00" class="time">14:00</a>
f13	policy is one thing, but how does one enforce policy?	<a href="#t14:00" class="time">14:00</a>
warren	f13, before brew, beehive would reject builds if N-V-R went backwards.	<a href="#t14:01" class="time">14:01</a>
warren	We could do something like that.	<a href="#t14:01" class="time">14:01</a>
wwoods	at some point we should have an rpm-diff tool running for proposed updates	<a href="#t14:02" class="time">14:02</a>
wwoods	and that's something it should test for	<a href="#t14:02" class="time">14:02</a>
warren	f13, I don't see anything wrong with enforcing N-V-R progression in the buildsystem itself.	<a href="#t14:02" class="time">14:02</a>
wwoods	but yeah, putting that into the buildsys works	<a href="#t14:03" class="time">14:03</a>
mclasen	warren: you mean E-V-R ?	<a href="#t14:03" class="time">14:03</a>
f13	ok, so we set policy now, and ask for koji enhancements to block a build of nevr goes backwards, for some value of backwards.	<a href="#t14:04" class="time">14:04</a>
warren	"N-V-R ADVANCEMENT ERROR: foo-1.2.3-2.fc8 is older than foo-1.2.4-1.fc8, please consider releasing a new version upstream or use an Epoch."	<a href="#t14:04" class="time">14:04</a>
warren	mclasen, yeah	<a href="#t14:04" class="time">14:04</a>
f13	Proposal:  Recommend to FESCo policy that upgrade path never be compromised either by removing builds or NEVR regressions.  File RFE in Koji to enforce rule at build time	<a href="#t14:05" class="time">14:05</a>
warren	"E-V-R ADVANCEMENT ERROR: foo-1.2.5-1.fc7 is newer than foo-1.2.4-1.fc8, please upgrade the version in F8 before F7."	<a href="#t14:06" class="time">14:06</a>
notting	sorry. gotta head out	<a href="#t14:06" class="time">14:06</a>
mclasen	f13: does that affect our flexibility wrt to building updates in different tags, if they inherit from each other ?	<a href="#t14:07" class="time">14:07</a>
f13	mclasen: example?	<a href="#t14:07" class="time">14:07</a>
warren	mclasen, you mean FC6, F7 and devel for example?	<a href="#t14:07" class="time">14:07</a>
mclasen	yeah	<a href="#t14:07" class="time">14:07</a>
f13	warren: I'm not quite sure I want to do forward blocking yet.	<a href="#t14:07" class="time">14:07</a>
warren	mclasen, version-release-disttag should be sufficient to prevent problems	<a href="#t14:08" class="time">14:08</a>
f13	warren: because it's more important to get a security update out for a releaesd product than it is for rawhide.	<a href="#t14:08" class="time">14:08</a>
mclasen	I was thinking about a situation where I build something for f7 first, and then my next devel build has to be newer than that	<a href="#t14:08" class="time">14:08</a>
mclasen	probably not a problem in practise	<a href="#t14:08" class="time">14:08</a>
warren	f13, if both F7 and F8 are out as stable distros that might become a good idea.	<a href="#t14:08" class="time">14:08</a>
f13	warren: that's more enforcement at update push time, rather than build time	<a href="#t14:08" class="time">14:08</a>
warren	mclasen, I'm thinking the buildsys only enforces forward E-V-R progression for a "stable" distro build if both distros are stable.	<a href="#t14:08" class="time">14:08</a>
f13	warren: lets not dictate what order a maintainer builds things in.	<a href="#t14:09" class="time">14:09</a>
warren	f13, I don't see a problem with doing it between two stable releases, when F7 and F8 are both released.	<a href="#t14:09" class="time">14:09</a>
f13	warren: and really, upgrading from a releaed f7 to a released f8 is going to have issues anyway	<a href="#t14:09" class="time">14:09</a>
warren	sure, but there shouldn't be any reason why a package in F8 should be < than F7	<a href="#t14:09" class="time">14:09</a>
f13	becuase currently your upgrade to f8 via supported methods is anaconda, and you can't add update repos then.	<a href="#t14:10" class="time">14:10</a>
f13	warren: at release of hte udpate time sure.  At build time, no.	<a href="#t14:10" class="time">14:10</a>
warren	f13, how is this a problem really?	<a href="#t14:10" class="time">14:10</a>
f13	warren: what right do we have to dictate what order people do their builds in?	<a href="#t14:10" class="time">14:10</a>
warren	f13, most cases where N+1 < N in extras were accidents.	<a href="#t14:10" class="time">14:10</a>
warren	f13, OK, I guess we can consider this later.	<a href="#t14:11" class="time">14:11</a>
f13	so long as the update requests land at the right time I'm happy.  I could give a shit what order they built the updates in.	<a href="#t14:11" class="time">14:11</a>
jwb_gone	f13, there's nothing to prevent fc-6 builds from going higher	<a href="#t14:11" class="time">14:11</a>
jwb_gone	it doesn't use koji	<a href="#t14:11" class="time">14:11</a>
f13	jwb_gone: not forever	<a href="#t14:11" class="time">14:11</a>
f13	so I didn't see any votes on the last proposal	<a href="#t14:12" class="time">14:12</a>
f13	<f13> Proposal:  Recommend to FESCo policy that upgrade path never be compromised either by removing builds or NEVR regressions.  File RFE in Koji to enforce rule at build time	<a href="#t14:12" class="time">14:12</a>
* bpepple gives a +1 from the peanut gallery.		<a href="#t14:13" class="time">14:13</a>
warren	f13, what happened to the test2 part?	<a href="#t14:13" class="time">14:13</a>
warren	f13, or at least the mention the "get an exception"?	<a href="#t14:13" class="time">14:13</a>
warren	who grants exceptions?  who makes that choice?	<a href="#t14:13" class="time">14:13</a>
f13	warren: I adjusted it to be forever.	<a href="#t14:13" class="time">14:13</a>
wwoods	we were convinced that "forever" is workable and useful	<a href="#t14:14" class="time">14:14</a>
f13	warren: rel-eng grants exception, dealt with on case by case basis.	<a href="#t14:14" class="time">14:14</a>
warren	f13, ok.	<a href="#t14:14" class="time">14:14</a>
rdieter	f13: +1 proposal	<a href="#t14:14" class="time">14:14</a>
warren	f13, +1	<a href="#t14:14" class="time">14:14</a>
wwoods	+1	<a href="#t14:15" class="time">14:15</a>
f13	jeremy: ?	<a href="#t14:15" class="time">14:15</a>
f13	well, he was pretty happy with this before.	<a href="#t14:15" class="time">14:15</a>
f13	poelcat: Decision: Recommend to FESCo policy that upgrade path never be compromised either by removing builds or NEVR regressions.  File RFE in Koji to enforce rule at build time	<a href="#t14:16" class="time">14:16</a>
f13 changed the topic of #fedora-meeting to: RELENG-Meeting - Open Discussion		<a href="#t14:16" class="time">14:16</a>
f13	that's the end of the agenda.  We're over time.  ANy pressing things?	<a href="#t14:16" class="time">14:16</a>
wwoods	do we have an official (or draft) policy for updates-testing stuff?	<a href="#t14:17" class="time">14:17</a>
warren	f13, until we're able to enable more rel-eng people to do pushing, could we have pushing done a little more often?	<a href="#t14:18" class="time">14:18</a>
f13	wwoods: all we decided on last week wwas to allow maintainers to skip updates-testing if they wanted to.	<a href="#t14:18" class="time">14:18</a>
warren	f13, sometimes pushing has waited on 2-3 days for example.  That has to be frustrating to our contributors.	<a href="#t14:18" class="time">14:18</a>
f13	warren: yeah, I'm trying to do the best we can here.  Once a day isn't totally unreasonable.	<a href="#t14:18" class="time">14:18</a>
f13	um, since release of f7 I don't believe I've gone for more than 2 days without a push	<a href="#t14:19" class="time">14:19</a>
warren	As long as we don't go beyond 1 day I think we're OK.	<a href="#t14:19" class="time">14:19</a>
f13 changed the topic of #fedora-meeting to: <a href="http://fedoraproject.org/wiki/Communicate/FedoraMeetingChannel">http://fedoraproject.org/wiki/Communicate/FedoraMeetingChannel</a> -- Meetings often get logged -- see schedule in the wiki for next meeting		<a href="#t14:20" class="time">14:20</a>
f13	thanks all.	<a href="#t14:20" class="time">14:20</a>

Generated by irclog2html.py 2.3 by Marius Gedminas - find it at mg.pov.lt!