ReleaseEngineering/Meetings/2008-feb-04

Fedora Release Engineering Meeting :: Monday 2008-02-04

Alpha

• Tuesday the bit gets flipped
• http://fedoraproject.org/wiki/QA/9/AlphaTreeTesting is the test status page

Reversing the netapp streams

• scheduled for today--we're working on pushing it back after the alpha
• change setup so that the content we generate in PHX can be put on a netapp in PHX
• instead of rsynced to RDU to be put on a netapp in RDU, and then mirrored by netapp back out to PHX
• this will result in shorter times between compose and visible to the world for things like rawhide and updates
• may delay the rawhide/updates for a bit

gcc 4.3

• gcc4.3 is in the buildroots for dist-f9 now, and some fallout has happened, namely kernel.
• striving to complete a mass rebuild before beta release

perl-5.10.0

• Feature page: http://fedoraproject.org/wiki/Features/Perl510
• 300 perl dependent packages as a sample set
• these packages were the base for which virtually all of the other perl dependent packages depend on
• out of those 300 about 60 more to build
• once complete the plan is to merge the new perl and the rebuilt bits into rawhide
• Fedora is the place where perl-5.10 was really first tested/used and the development ground for getting all the upstreams ported to the new perl

IRC Transcript

-!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering Meeting		<a href="#t13:05" class="time">13:05</a>
f13	ping: notting jeremy jwb spot warren rdieter wwoods poelcat lmacken	<a href="#t13:06" class="time">13:06</a>
* notting is here		<a href="#t13:06" class="time">13:06</a>
* lmacken		<a href="#t13:06" class="time">13:06</a>
* jeremy is here		<a href="#t13:06" class="time">13:06</a>
* spot is here		<a href="#t13:07" class="time">13:07</a>
* poelcat here		<a href="#t13:07" class="time">13:07</a>
f13	aight.	<a href="#t13:08" class="time">13:08</a>
-!- f13 changed the topic of #fedora-meeting to: Fedora Release Engineering Meeting - F9 Alpha		<a href="#t13:08" class="time">13:08</a>
f13	so the Alpha is being staged by mirrors.  They've had access to it since Friday	<a href="#t13:08" class="time">13:08</a>
f13	so hopefully we'll have enough ready to go tomorrow	<a href="#t13:08" class="time">13:08</a>
f13	I'm in teh process of uploading the isos to the torrent server and creating the torrents.  That should be done this afternoon	<a href="#t13:08" class="time">13:08</a>
lmacken	f13: F9-Alpha-i686-20080129.0.iso ?	<a href="#t13:09" class="time">13:09</a>
f13	lmacken: yeah, but renamed.	<a href="#t13:09" class="time">13:09</a>
lmacken	cool.	<a href="#t13:09" class="time">13:09</a>
f13	we'll hope the bit gets flipped and all will be happy tomorrow.	<a href="#t13:10" class="time">13:10</a>
f13	anybody have anything else to bring up wrt Alpha?	<a href="#t13:10" class="time">13:10</a>
wwoods	<a href="http://fedoraproject.org/wiki/QA/9/AlphaTreeTesting">http://fedoraproject.org/wiki/QA/9/AlphaTreeTesting</a> is the test status page	<a href="#t13:11" class="time">13:11</a>
wwoods	we may need an updates.img for upgrades	<a href="#t13:11" class="time">13:11</a>
f13	yeah, it's a bit rough around the edges, but it is an Alpha	<a href="#t13:11" class="time">13:11</a>
wwoods	yup	<a href="#t13:11" class="time">13:11</a>
wwoods	just trying to keep ahead of bug reporters to make life easier for triagers/developers	<a href="#t13:12" class="time">13:12</a>
f13	nod	<a href="#t13:12" class="time">13:12</a>
wwoods	oh - I want to pick a week sometime this month	<a href="#t13:12" class="time">13:12</a>
wwoods	a couple weeks before the string freeze	<a href="#t13:12" class="time">13:12</a>
wwoods	to make double-damn sure we have working boot/install images	<a href="#t13:12" class="time">13:12</a>
wwoods	so that we can put out a call for translators to grab the rescuecd (or minimal-installer.iso or whatever we call it)	<a href="#t13:13" class="time">13:13</a>
jeremy	wwoods: *nod*	<a href="#t13:13" class="time">13:13</a>
jeremy	and I'm probably going to write a script in the next few days which will help make that a bit easier	<a href="#t13:13" class="time">13:13</a>
wwoods	probably that'd be a good time to also make sure we have good Live images so we can have a big internal Laptop Test-fest	<a href="#t13:14" class="time">13:14</a>
wwoods	jeremy: sweet	<a href="#t13:14" class="time">13:14</a>
wwoods	do we have ppc images for alpha?	<a href="#t13:15" class="time">13:15</a>
f13	wwoods: yes	<a href="#t13:15" class="time">13:15</a>
f13	not live ones though	<a href="#t13:15" class="time">13:15</a>
f13	sorry	<a href="#t13:15" class="time">13:15</a>
wwoods	's cool	<a href="#t13:16" class="time">13:16</a>
wwoods	I should probably do some ppc testing.. or ask jwb to update the test status page	<a href="#t13:17" class="time">13:17</a>
wwoods	f13: if you can give me a pointer to an internal copy of the ppc bits I'll follow up on that after this meeting	<a href="#t13:17" class="time">13:17</a>
f13	wwoods: it's on reducto	<a href="#t13:17" class="time">13:17</a>
f13	moving on...	<a href="#t13:19" class="time">13:19</a>
-!- f13 changed the topic of #fedora-meeting to: Fedora RelEng: Reversing the netapp streams		<a href="#t13:19" class="time">13:19</a>
notting	this was scheduled for today. we're working on pushing it back after the alpha	<a href="#t13:19" class="time">13:19</a>
f13	later this week we would like to reverse the netapp streams so that the content we generate in PHX can be put on a netapp in PHX, rather than rsynced to RDU to be put on a netapp in RDU, and then mirrored by netapp back out to PHX	<a href="#t13:19" class="time">13:19</a>
f13	this will result in shorter times between compose and visible to the world for things like rawhide and updates	<a href="#t13:20" class="time">13:20</a>
f13	a number of our processes will have to change though	<a href="#t13:20" class="time">13:20</a>
f13	and ti's possible that rawhide/updates may be delayed for a bit	<a href="#t13:21" class="time">13:21</a>
f13	as we trial and error our way through it	<a href="#t13:21" class="time">13:21</a>
f13	notting, lmacken, mmcgrath, and I are the primary folks on this task, so if you have any thoughts/concerns/suggestions hit up one of us.	<a href="#t13:22" class="time">13:22</a>
f13	anything further on this subject?	<a href="#t13:23" class="time">13:23</a>
f13	aight.	<a href="#t13:23" class="time">13:23</a>
-!- f13 changed the topic of #fedora-meeting to: Fedora RelEng: Gcc43		<a href="#t13:23" class="time">13:23</a>
f13	gcc4.3 is in the buildroots for dist-f9 now, and some fallout has happened, namely kernel.	<a href="#t13:24" class="time">13:24</a>
f13	but more importantly is that jakub has indicated that he'd like to see a massrebuild of our packages against the new gcc at some point, after a week or so if it being in the buildroots	<a href="#t13:24" class="time">13:24</a>
notting	so... next week?	<a href="#t13:25" class="time">13:25</a>
f13	March 4th is the Beta freeze	<a href="#t13:25" class="time">13:25</a>
f13	so we'd likely have to do it next week or the week after	<a href="#t13:25" class="time">13:25</a>
f13	and we have to decide if htis is going to be maintainer driven, or releng driven.	<a href="#t13:25" class="time">13:25</a>
f13	IE we ask maintainers to do it and wait, or we just plow through and script massive amounts of builds.	<a href="#t13:25" class="time">13:25</a>
notting	automated seems simpler to me	<a href="#t13:26" class="time">13:26</a>
f13	you'd think.	<a href="#t13:26" class="time">13:26</a>
notting	esp. if there aren't ordering concerns	<a href="#t13:26" class="time">13:26</a>
f13	trying not to have conflicts is the problem.	<a href="#t13:26" class="time">13:26</a>
f13	IE automation kicks off a build, maintainer comes along after that and also kicks off a build.	<a href="#t13:27" class="time">13:27</a>
f13	automated was a backgroundjob, so the maintainer's job gets done first, then the automated one, automated (older) one is now the last tagged, and thus the 'newest'.	<a href="#t13:27" class="time">13:27</a>
poelcat	f13: wouldn't we want the rebuilt packages to be in the beta?	<a href="#t13:27" class="time">13:27</a>
f13	poelcat: yeah... that's why we're doing it before the beta freeze	<a href="#t13:28" class="time">13:28</a>
poelcat	f13: sorry misread above	<a href="#t13:28" class="time">13:28</a>
notting	f13: is there a way to not do 'automated/background'? or to have koji check for backwardsness?	<a href="#t13:29" class="time">13:29</a>
f13	notting: if we don't do background, we essentially tie up the buildsystem for a few days.  no other buidls will get through until our entire set is done.	<a href="#t13:30" class="time">13:30</a>
warren_treo	sorry at doctor	<a href="#t13:30" class="time">13:30</a>
f13	notting: as for backwardness, I don't thikn there is anything koji itself can do.  We can run a query on the tag after all is said and done to detect nvr issues though.	<a href="#t13:30" class="time">13:30</a>
f13	the added fun is that spot is going to want to land the new perl relatively soon as well, which is going to cause all kinds of pain	<a href="#t13:31" class="time">13:31</a>
warren_treo	what ver of perl?	<a href="#t13:31" class="time">13:31</a>
spot	5.10.0	<a href="#t13:31" class="time">13:31</a>
warren_treo	wow	<a href="#t13:31" class="time">13:31</a>
* spot has been chainbuilding for weeks		<a href="#t13:32" class="time">13:32</a>
warren_treo	spot: is there a URL describing the compat impact of this new perl?	<a href="#t13:33" class="time">13:33</a>
* warren_treo wonders if anybody tested spamassassin on perl 5.10		<a href="#t13:34" class="time">13:34</a>
notting	f13: so, do we have the scripts ready to start the automated buildatron?	<a href="#t13:35" class="time">13:35</a>
f13	notting: nope.  I haven't spent any time on that recently	<a href="#t13:35" class="time">13:35</a>
warren_treo	build order is known?	<a href="#t13:35" class="time">13:35</a>
f13	warren_treo: we're ignoring buildorder	<a href="#t13:36" class="time">13:36</a>
warren_treo	oh	<a href="#t13:36" class="time">13:36</a>
f13	at least for the first pass.	<a href="#t13:36" class="time">13:36</a>
f13	to avoid cvs conflicts, I imagine the script would go through each needed build (query to fall out of koji for things not yet built with new gcc), do a checkout, do an autobump of hte spec (*shudder*), commit/tag/build --background; and move on.	<a href="#t13:37" class="time">13:37</a>
notting	when was the last time we did one?	<a href="#t13:38" class="time">13:38</a>
f13	most of that has been done before	<a href="#t13:38" class="time">13:38</a>
f13	the autospecbump does run into problems every now and again with malformed spec files	<a href="#t13:38" class="time">13:38</a>
f13	and catching either cvs errors, or tag errors can be difficult, as well as gathering in the build failures for investigation.	<a href="#t13:38" class="time">13:38</a>
warren_treo	malformed as in encoding.	<a href="#t13:39" class="time">13:39</a>
warren_treo	?	<a href="#t13:39" class="time">13:39</a>
f13	warren_treo: no, as in dirty games with Release: tags	<a href="#t13:39" class="time">13:39</a>
f13	programatically editing spec files is not easy	<a href="#t13:39" class="time">13:39</a>
notting	was the last rebuild maintainer-driven?	<a href="#t13:39" class="time">13:39</a>
warren_treo	nod	<a href="#t13:39" class="time">13:39</a>
f13	notting: yeah	<a href="#t13:39" class="time">13:39</a>
notting	the other option is to punt maintainer vs. automated to fesco >:)	<a href="#t13:40" class="time">13:40</a>
warren_treo	btw did we get past the gcc43 kernel build issue?	<a href="#t13:40" class="time">13:40</a>
f13	the last one that we did any scripting of was fc6 time frame.  And then some smaller less scripted ones for the last glibc fallout where ppc was boned.	<a href="#t13:40" class="time">13:40</a>
f13	warren_treo: not yet afaict	<a href="#t13:40" class="time">13:40</a>
f13	also, last time we let maintainers opt in/out of autobuilding	<a href="#t13:41" class="time">13:41</a>
f13	usually with a file in their module.	<a href="#t13:41" class="time">13:41</a>
f13	something we should consider this time.	<a href="#t13:41" class="time">13:41</a>
f13	basically releng should come up with a proposal on how to accomplish the rebuilds and present it to FESCo	<a href="#t13:42" class="time">13:42</a>
warren_treo	I hope we can do it without cvs...	<a href="#t13:42" class="time">13:42</a>
f13	I suppose that will fall on me unless anybody else wants to take a stab at it.	<a href="#t13:42" class="time">13:42</a>
warren_treo	optin out that is	<a href="#t13:42" class="time">13:42</a>
f13	warren_treo: have better ideas?	<a href="#t13:42" class="time">13:42</a>
f13	we need to identify and mark packages that need to be rebuilt	<a href="#t13:42" class="time">13:42</a>
f13	and then let maintainers scrub them from the list.	<a href="#t13:43" class="time">13:43</a>
f13	maybe a wiki page?	<a href="#t13:43" class="time">13:43</a>
warren_treo	could it be added as a flag in pkgdb?  might be quick	<a href="#t13:43" class="time">13:43</a>
notting	pkgdb lives forever, though	<a href="#t13:43" class="time">13:43</a>
warren_treo	mm	<a href="#t13:43" class="time">13:43</a>
f13	yeah, not sure if pkgdb is setup for arbitrary flags like that	<a href="#t13:43" class="time">13:43</a>
f13	so I'll work up a plan, and ping ya'll as it comes along.  Not enough days in the week.	<a href="#t13:45" class="time">13:45</a>
-!- f13 changed the topic of #fedora-meeting to: Fedora RelEng: perl-5.10.0		<a href="#t13:45" class="time">13:45</a>
warren_treo	prior to gcc43 I heard mention that mass rebuild wouldn't be necessary, what happened?	<a href="#t13:45" class="time">13:45</a>
f13	warren_treo: it's not /necessary/, but desired by the gcc team.	<a href="#t13:46" class="time">13:46</a>
warren_treo	ok	<a href="#t13:46" class="time">13:46</a>
f13	spot: can you give us a rundown on the perl status?	<a href="#t13:46" class="time">13:46</a>
spot	sure.	<a href="#t13:46" class="time">13:46</a>
spot	i identified 300 perl dependent packages as a sample set	<a href="#t13:46" class="time">13:46</a>
spot	these packages were the base for which virtually all of the other perl dependent packages depend on	<a href="#t13:47" class="time">13:47</a>
spot	Out of those 300	<a href="#t13:47" class="time">13:47</a>
spot	i have about 60 more to build	<a href="#t13:47" class="time">13:47</a>
spot	once that is done, the plan is to merge the new perl and the rebuilt bits into rawhide	<a href="#t13:48" class="time">13:48</a>
warren_treo	perl-5.10 and pkgs built against it are in a separate tag now?	<a href="#t13:48" class="time">13:48</a>
spot	warren: yes, dist-f9-perl	<a href="#t13:48" class="time">13:48</a>
spot	once merged, there will still be lots of things broken	<a href="#t13:49" class="time">13:49</a>
f13	spot: are you planning on doing a Feature for perl-5.10, which includes info about when it'll land, and what we expect other perl maintianers to do once it does land?	<a href="#t13:50" class="time">13:50</a>
f13	(and a rollback plan)	<a href="#t13:50" class="time">13:50</a>
spot	f13: Yep, just hadn't gotten a round tuit yet.	<a href="#t13:50" class="time">13:50</a>
notting	spot: why will there be things broken?	<a href="#t13:50" class="time">13:50</a>
warren_treo	rollback sounds scary when hundreds of pkgs re involved :)	<a href="#t13:50" class="time">13:50</a>
f13	warren_treo: untag!	<a href="#t13:51" class="time">13:51</a>
f13	unless you want to force epoch into all of those.... (:	<a href="#t13:51" class="time">13:51</a>
spot	notting: because everything hasn't been rebuilt against the new perl yet	<a href="#t13:51" class="time">13:51</a>
notting	spot: .... is dist-f9-perl not going to encompass 'everything'?	<a href="#t13:51" class="time">13:51</a>
warren_treo	even things like pidgin build against perl	<a href="#t13:51" class="time">13:51</a>
notting	or are you only rebuilding pure-perl	<a href="#t13:52" class="time">13:52</a>
notting	?	<a href="#t13:52" class="time">13:52</a>
spot	notting: it would be much faster if the individual maintainers did the rebuilds	<a href="#t13:52" class="time">13:52</a>
* warren_treo wonders what will utterly fail		<a href="#t13:52" class="time">13:52</a>
spot	warren: so far, only one thing has failed, and I fixed it.	<a href="#t13:52" class="time">13:52</a>
f13	this could put Fedora in a pretty cool place	<a href="#t13:52" class="time">13:52</a>
warren_treo	spot: what was the cause of that failure?	<a href="#t13:52" class="time">13:52</a>
spot	the difficulty is in the ordering	<a href="#t13:52" class="time">13:52</a>
f13	Fedora was the place where perl-5.10 was really first tested/used and the development ground for getting all the upstreams ported to the new perl.	<a href="#t13:53" class="time">13:53</a>
spot	warren: old crufty code making bad assumptions about array behavior	<a href="#t13:53" class="time">13:53</a>
warren_treo	f13: yes assuming the code continues to work	<a href="#t13:53" class="time">13:53</a>
warren_treo	oh	<a href="#t13:53" class="time">13:53</a>
notting	f13: 'fedora 9 shipped a new perl. it worked. no one noticed.'?	<a href="#t13:53" class="time">13:53</a>
warren_treo	gotta go now, appt coming up	<a href="#t13:54" class="time">13:54</a>
spot	notting: upstream perl certainly is watching this.	<a href="#t13:54" class="time">13:54</a>
spot	we've got their backing as we go forward	<a href="#t13:54" class="time">13:54</a>
spot	i did the rebuild of these 300 packages so that the maintainers could just rebuild their bits without having to hunt down the other deps in the chain	<a href="#t13:55" class="time">13:55</a>
spot	this is the trunk. everything else is leaves.	<a href="#t13:55" class="time">13:55</a>
f13	the day we merge, is the broken dep report going to be 400 packages long?	<a href="#t13:55" class="time">13:55</a>
spot	yes.	<a href="#t13:55" class="time">13:55</a>
f13	hawt	<a href="#t13:56" class="time">13:56</a>
spot	but, for those folks tracking rawhide, it should still be mostly upgradeable.	<a href="#t13:56" class="time">13:56</a>
notting	spot: is it possible to spit out a compat-libperl for the interim?	<a href="#t13:57" class="time">13:57</a>
spot	notting: nope.	<a href="#t13:57" class="time">13:57</a>
spot	i really don't want to do that.	<a href="#t13:57" class="time">13:57</a>
notting	spot: how many of the broken leaves are in the build chain?	<a href="#t13:57" class="time">13:57</a>
spot	in the build chain for other, non-perl packages?	<a href="#t13:58" class="time">13:58</a>
notting	yes. say, auto*	<a href="#t13:58" class="time">13:58</a>
spot	nada.	<a href="#t13:58" class="time">13:58</a>
spot	the 300 rebuilds cover those cases.	<a href="#t13:58" class="time">13:58</a>
notting	ok then. carry on, sir!	<a href="#t13:58" class="time">13:58</a>
spot	(base perl pulled in a lot of external modules in 5.10.0)	<a href="#t13:58" class="time">13:58</a>
spot	made things somewhat simpler.	<a href="#t13:59" class="time">13:59</a>
spot	a few packages will have to be blocked in devel at the same time that we merge	<a href="#t13:59" class="time">13:59</a>
spot	i've identified those (still need to email their maintainers and let them know)	<a href="#t13:59" class="time">13:59</a>
nirik	couldn't you just add that dist-f9-perl tag to the inheritance in koji, and rawhide would pick up all those builds? or were they just testing builds?	<a href="#t13:59" class="time">13:59</a>
spot	nirik: that's certainly one way to do it. i defer to f13 on how he wants to do it.	<a href="#t14:00" class="time">14:00</a>
spot	we need to block those packages because their n-v-r might be higher than what is in base perl.	<a href="#t14:00" class="time">14:00</a>
f13	nirik: at the end of the day it doesn't necessarily matter.	<a href="#t14:00" class="time">14:00</a>
f13	actually wait	<a href="#t14:00" class="time">14:00</a>
f13	it does matter.	<a href="#t14:01" class="time">14:01</a>
nirik	well, it seems a waste to rebuild them all... but whatever works.	<a href="#t14:01" class="time">14:01</a>
f13	a number of these perl packages have already been built in dist-f9	<a href="#t14:01" class="time">14:01</a>
f13	nirik: they won't be rebuilt.	<a href="#t14:01" class="time">14:01</a>
f13	we can't add dist-f9-perl to the inheritance chain, as anythign expecitly built in dist-f9 will win over anythign inherited, regardless of nvr	<a href="#t14:01" class="time">14:01</a>
f13	instead, we can just koji tag-pkg everything in dist-f9-perl to dist-f9.	<a href="#t14:01" class="time">14:01</a>
f13	no rebuilding, then everything has a dist-f9 tag.	<a href="#t14:02" class="time">14:02</a>
nirik	cool.	<a href="#t14:02" class="time">14:02</a>
spot	yeah, i wasn't planning on doing two rounds of rebuilds. :)	<a href="#t14:02" class="time">14:02</a>
f13	spot: you'll still have to identify the things that have been built since you did your dist-f9-perl build of them	<a href="#t14:02" class="time">14:02</a>
spot	yep.	<a href="#t14:02" class="time">14:02</a>
spot	thankfully, most of these things don't move much at all.	<a href="#t14:03" class="time">14:03</a>
f13	nod	<a href="#t14:03" class="time">14:03</a>
f13	ok, we're running out of time here.	<a href="#t14:03" class="time">14:03</a>
f13	spot: do you want to try and land perl before or after the mass rebuild?	<a href="#t14:03" class="time">14:03</a>
spot	perfect world, before.	<a href="#t14:04" class="time">14:04</a>
f13	(and I imagine a number of your packages will have to get built anyway, for the new gcc)	<a href="#t14:04" class="time">14:04</a>
f13	ok.	<a href="#t14:04" class="time">14:04</a>
spot	f13: some of them have already picked up the new gcc	<a href="#t14:04" class="time">14:04</a>
spot	it doesn't seem to break anything yet.	<a href="#t14:04" class="time">14:04</a>
notting	break-everything-at-once?	<a href="#t14:04" class="time">14:04</a>
f13	notting: please no.	<a href="#t14:04" class="time">14:04</a>
spot	i did a rebuild of perl-5.10 itself after gcc43 hit rawhide	<a href="#t14:04" class="time">14:04</a>
spot	and it is fine.	<a href="#t14:04" class="time">14:04</a>
f13	spot: goot.	<a href="#t14:04" class="time">14:04</a>
f13	spot: lets see where you're at next week?  I"d like to see FESCo get some visibility on this beofre we start breaking the world.	<a href="#t14:05" class="time">14:05</a>
spot	sure.	<a href="#t14:05" class="time">14:05</a>
spot	hopefully, i'll be ready before then.	<a href="#t14:05" class="time">14:05</a>
notting	well, that would push back the 'regular' mass rebuild, then	<a href="#t14:06" class="time">14:06</a>
notting	(which is ok, just making sure we have a plan together)	<a href="#t14:06" class="time">14:06</a>
f13	notting: yeah, perl next week, mass rebuild the week after, leading up to Beta	<a href="#t14:06" class="time">14:06</a>
-!- f13 changed the topic of #fedora-meeting to: Fedora RelEng: Open Discussion		<a href="#t14:09" class="time">14:09</a>
f13	any thing else people would like to talk about this week?	<a href="#t14:09" class="time">14:09</a>
f13	alright, calling it a meeting.	<a href="#t14:12" class="time">14:12</a>
f13	cheers all!	<a href="#t14:12" class="time">14:12</a>

Generated by irclog2html.py 2.3 by Marius Gedminas - find it at mg.pov.lt!