Infrastructure/SCMSig/Meeting-2007-07-26

[14:01]  jcollie has set the subject to SCM Sig starting... give everyone a minute or so to show up
[14:01]  abadger1999: So jwb couldn't make it but I know he had a lot of reasons he didn't like it.
[14:02]  abadger1999: s/it/exploded trees/
[14:02]  abadger1999: We'll have to remember to ping him afterwards.
[14:02]  jcollie: yeah... i've been updating the agenda: http://fedoraproject.org/wiki/Infrastructure/SCMSig/Agenda
[14:02]  mmcgrath: jcollie: has anyone responded to you at all?
[14:02]  jcollie: and it seemed i was adding more problems/pitfalls
[14:03]  paulobanon: mmcgrath: thats the biggest question
[14:03]  jcollie: mmcgrath, there was one followup on fedora-devel list to abadger1999's announcement
[14:03]  mmcgrath: thats good then.
[14:03]  abadger1999: nim-nim had issues from the previous mailing list thread but I haven't had time to go back and add them to the wiki.
[14:04]  jcollie: ok, so let's get started...
[14:04]  jcollie has set the subject to SCM Sig Meeting - "Exploded Trees"
[14:04]  mmcgrath: kaboom!
[14:04]  walters has left ("Ex-Chat" (n=walters@static-71-243-117-136.bos.east.verizon.net))
[14:04]  jcollie: role call... I'm here obviously
[14:04]  * mmcgrath here
[14:04]  paulobanon: here
[14:04]  * jbowes is here
[14:05]  cebbert has left ( (i=cebbert@nat/redhat/x-84a37e3193eab2da))
[14:05]  chjunaidanwar: here
[14:05]  * abadger1999 here
[14:05]  jcollie: ok, well the purpose of this meeting is to decide whether exploded trees offer enough of a benefit to spend the time figuring out how to work around all the issues that will crop up
[14:06]  jcollie: http://fedoraproject.org/wiki/Infrastructure/SCMSig/Agenda
[14:06]  jcollie: is what i was working on
[14:06]  jcollie: before we discuss the problems, are there any features/benefits anyone wants to add?
[14:06]  * mmcgrath is against exploded trees but that could be because I'm not educated enough in its theory to understand it.
[14:06]  jcollie: well the theory is easy enough
[14:07]  f13: I like the theory of Exploaded trees used as a tool to help wth patch management, but the end result in srpms and such is a prestine tarball + a set of patches.
[14:07]  mmcgrath: so you store everything thats needed to build the RPM in the source tree?
[14:07]  jcollie: instead of having .spec + tarball + patches in your SCM, you have the exploded source checked into your scm
[14:07]  abadger1999: f13: Agreed.
[14:07]  jcollie: mmcgrath, yeah
[14:07]  f13: and the trees would be optional.  Those that don't deal with patches wouldn't ever have to see them.
[14:07]  mmcgrath: does that sound like a management mess to anyone?
[14:08]  jcollie: yeah it could be
[14:08]  skvidal: mmcgrath: yes
[14:08]  abadger1999: mmcgrath: Maybe maybe not.  When actively hacking on something, people use a vendor branch all the time.
[14:08]  jcollie: we would definitely have to discipline maintainers into keeping branches with various changes
[14:08]  f13: see, I think you'd /still/ have spec + patches in scm, tarball in lookaside, and optional exploaded source/ as a subdir.
[14:08]  skvidal: mmcgrath: now add dvcs on top of that and we end up having to constantly figure out what branch someone is referring to
[14:09]  mmcgrath: I guess I mean to ask the question this way.
[14:09]  mmcgrath: Will this benefit the top 1% of our contributors while screwing everyone else?
[14:09]  abadger1999: mmcgrath: But we *shouldn't* be hacking as much as that so it may not be a valid comparison.
[14:09]  f13: (looks like I got sucked into this after all)
[14:09]  f13: mmcgrath: it wouldn't have to.
[14:09]  jcollie: f13, heh heh
[14:09]  skvidal: f13: umm, wouldn't _have to_
[14:09]  f13: mmcgrath: in my vision the only people who would ever see this are those that run a command to get a checkout of the source to play with
[14:10]  jcollie: yeah, if we had a convention on what went into which branches and people actually stuck to it...
[14:10]  * lennert here
[14:10]  f13: everybody else would continue to see things like they see it today, spec (+ patches), tarball in lookaside.
[14:10]  jbowes: f13: so that's not really exploded source in the scm, so much as maybe convenience makefile targets?
[14:10]  skvidal: f13: how is this different than a 'make  explodetree' in our current system
[14:10]  mmcgrath: f13: so basically, its identical to what we have now.  We'd just be encouraging more people to stick stuff in the VCS?  (like I store .htaccess files, default configs, READMES in cvs now)
[14:10]  jcollie: no, it's completely different
[14:11]  f13: skvidal: jbowes; the convenience is that it would be in the /same/ scm as the spec and patches sothat you can use scm tools to deal with them.
[14:11]  skvidal: f13: huh?
[14:11]  skvidal: if I have a tarball in the lookaside
[14:11]  abadger1999: f13: If we went that route, the exploded tree has to be persistent, though.
[14:11]  skvidal: and I type (in theory)
[14:11]  jbowes: f13: so like make explodetree plus commit to scm
[14:11]  skvidal: jbowes: which means we now have 2 copies of that data - one of the uncompressed w/an assload of metadata
[14:11]  jcollie: skvidal, in the "pure" exploded tree concept there wouldn't be tarballs in a lookaside
[14:11]  abadger1999: So kernel maintainer runs make explodetree once and then the kernel checkout always uses an exploded tree.
[14:12]  skvidal: jcollie: WOAH
[14:12]  f13: jbowes: sure for the simplistic sake.  If there were some persistance then you wouldn't have to repeat actions allthe time.
[14:12]  abadger1999: Otherwise most of the benefits go away.
[14:12]  skvidal: we have to have tarballs
[14:12]  skvidal: we have to be able to verify pristine source
[14:12]  f13: actually yea, lets clear something up righ tnow.
[14:12]  jcollie: skvidal, yeah it's a really radical concept
[14:12]  skvidal: jcollie: WE HAVE TO VERIFY PRISTINE SOURCE
[14:12]  f13: Proposal; no matter what, srpms continue to be prestine source + patches.  Period.
[14:12]  jcollie: f13, yeah that's one of the pitfalls i mention
[14:12]  f13: +1
[14:12]  abadger1999: skvidal: I agree.  We'll have to have two copies of the source.
[14:13]  skvidal: abadger1999: so why?
[14:13]  skvidal: why do that
[14:13]  abadger1999: lookaside + exploded tree.
[14:13]  skvidal: why not just have it explode a tree on the fly
[14:13]  chjunaidanwar: +1
[14:13]  jbowes: f13: +1
[14:13]  skvidal: when a developer wants it
[14:13]  skvidal: like we do with a mockbuild
[14:13]  skvidal: make explodetree in 'devel' off of yum
[14:13]  abadger1999: Because one of the benefits of exploded tree is to be able to see your history as you modify and rebase a patch.
[14:13]  abadger1999: If you get rid of the tree between checkouts then you don't have the history.
[14:14]  skvidal: how much patching are we talking about?
[14:14]  jcollie: skvidal, i think what could make life easier is to make it easy for a maintainer to set up a public repository to develop patches in
[14:14]  lennert: you can use stuff like interdiff to see diffs between patches
[14:14]  lennert: i.e. maybe it's just a matter of some extra scripting
[14:14]  mmcgrath: Just to give you guys an idea in terms of size:
[14:14]  mmcgrath: 3.6G    /cvs/pkgs/
[14:14]  mmcgrath: 123G    /repo/pkgs/
[14:14]  jcollie: is repo/pkgs the lookaside?
[14:14]  f13: uh, what are these?
[14:14]  skvidal: jcollie: what does that buy a maintainer?
[14:15]  skvidal: jcollie: to have a repo of patches - and how are we stopping them from doing that, now.
[14:15]  jcollie: skvidal, coordination with upstream, comaintainers, downstream
[14:15]  f13: I think maybe we need to look at how many of our packages have more than say 2 or 3 patches to see what percentage of our base we're helping.
[14:15]  JSchmitt has joined the group chat (n=s4504kr@p54B11BFA.dip0.t-ipconnect.de)
[14:15]  skvidal: f13: more than 2 or 3 patches > 1K
[14:16]  jcollie: skvidal, sure someone could set up their own repo, what i'm saying is that we'd make it easy to host on fedora infrastructure
[14:16]  f13: skvidal: point.
[14:16]  mmcgrath: jcollie: yeah
[14:16]  skvidal: f13: one line hacklets to an init script don't really matter
[14:16]  lennert: i have ~50 patches in my kernel tree at any given moment, but i still keep it in svn as a quilt queue.. because i want to be able to distinguish where stuff came from, etc.
[14:16]  abadger1999: lennert: Possibly.  But it goes beyond interdiff.  Maybe quilt + interdiff + integration to cvs would be how to visualise the desired features.
[14:16]  mmcgrath:  /repo/pkgs is all of our lookaside (zipped)   /cvs/pkgs is all of our cvs (unzipped)
[14:16]  skvidal: lennert: an let's be honest kernel is an exception
[14:16]  lennert: to me it suonds like exploded trees is trying to solve a problem that can also be solved with some scripting
[14:16]  abadger1999: integrattion to cvs meaning: transparently pull and interdiff between past revisions of a patch.
[14:16]  jcollie: skvidal, that way it's easier to find someone's repo, and everyone doesn't have to run around setting up git/hg web interfaces etc
[14:16]  jbowes: mmcgrath: does the lookaside keep old releases?
[14:16]  lennert: skvidal: well, many patches against non-kernel packages are long-lived as well, or at least, that's my impression
[14:17]  mmcgrath: jbowes: yeah, I *think* we have to keep all sources around for 3 years.  (I may have made this up)  but right now we keep all sources forever.
[14:17]  skvidal: jbowes: which exploded trees would do, too.
[14:18]  jbowes: yeah.
[14:18]  skvidal: but now we're doubling that
[14:18]  jcollie: so if i'm hearing right, "pure" expoded tree-style package management is a non-starter because of the inability to verify against upstream tarballs
[14:18]  f13: yes
[14:18]  skvidal: jcollie: yes
[14:18]  mmcgrath: jcollie: and possibly not having enough storage on the server.
[14:19]  f13: jeremy: are you watching this at all?
[14:19]  jcollie: mmcgrath, actually i think that storage in a "pure" fashion could be smaller but that's not the real concern
[14:19]  mmcgrath: jcollie: you may be right.
[14:20]  abadger1999: blizzard: as well (if you're not still running around with the new baby.)
[14:20]  f13: abadger1999: I just pinged him elsewhere (:
[14:21]  jcollie: ok, so what about what i was talking about before... the main repository for a package would still contain spec, tarball & patches but we make it easy for maintainers to host a repository on a fedora host?
[14:22]  skvidal: jcollie: host how?
[14:22]  skvidal: jcollie: it wouldn't be writable
[14:22]  skvidal: not unless the other players had fas accts
[14:22]  jcollie: oh yeah, it'd only be writable by people with fas accounts
[14:22]  JSchmitt has left ("Konversation terminated!" (n=s4504kr@fedora/JSchmitt))
[14:23]  skvidal: jcollie: so how is that different from now, really?
[14:23]  skvidal: I mean you, as a maintainer, can check in whatever you want, really.
[14:23]  jcollie: right now i don't think we encourage exploding the whole source into CVS
[14:23]  skvidal: there's nothing stopping you from doing it, though.
[14:23]  abadger1999: I think someone would kill me if I checked in a source tree
[14:23]  jcollie: true
[14:23]  skvidal: but that's just it
[14:23]  skvidal: if someone would kill you now
[14:23]  abadger1999: wrath of racor.
[14:24]  skvidal: why do we want to make it okay in the new system?
[14:24]  jcollie: communication
[14:24]  jeremy: f13: in another discussion... thought this wasn't until 4 for some reason
[14:24]  skvidal: have the things that make someone want to kill you gone away?
[14:25]  jcollie: basically, developing your patches in a public repo make it easier to communicate changes
[14:25]  skvidal: jcollie: make sources; untar; cvs add dir-you-just-untarred; cvs ci
[14:25]  jcollie: yeah, but i'd like to keep that out of the main repo
[14:25]  skvidal: jcollie: so if we go to a non-cvs scm
[14:25]  skvidal: and it's available via anon-http or $scm-daemon
[14:25]  skvidal: you just want it to not be a no-no to explode your tree?
[14:26]  jcollie: into the main package repository yeah
[14:26]  abadger1999: Features: patch history.  ability to easily target which patch a change should be made to.  scripted creation of vendor branches. Using the SCM to aid rebasing.
[14:26]  skvidal: so if we make it 'ok' for a maintainer to explode a tree
[14:26]  jcollie: yeah, plus if we host the repository we can teach the makefiles to extract patches from the secondary repository
[14:26]  abadger1999: All those would be filled by skvidal's last statement.
[14:27]  skvidal: doesn't that solve your need jcollie
[14:27]  skvidal: and then we don't need to do this to everyone
[14:27]  skvidal: if you maintain packages which have 3, 4 line patches, for example.
[14:27]  skvidal: and you're mostly a packager of an upstream that "just works"
[14:28]  jcollie: no, i wasn't suggesting that everyone would have to do that
[14:28]  skvidal: if I'm waaaaaaaaaaaaay out in left field tell me
[14:28]  lennert: you could make the exploded tree read-only, and auto-generated from the rw non-exploded tree
[14:28]  ajax has joined the group chat (i=ajackson@nat/redhat/x-1a0cf3da10d71e28)
[14:28]  lennert: what you want to avoid is people committing to the exploding tree and then losing info about what change came from where
[14:28]  jcollie: in my view koji would still be expecting spec, tarball, patch files in the main repository
[14:28]  f13: I asked ajax to pop in as I think he would have some interesting perspective of how our current workflow is not good and how it could improve.
[14:29]  lennert: so maybe you shouldn't let people commit to the exploded tree at all then
[14:29]  f13: ajax: right now we're discussing the idea of somehow having an exploaded tree ofthe upstream source to play with
[14:29]  abadger1999: lennert: Uhm... no.  You need to have logic in your scm tools to manage that.
[14:29]  * mmcgrath still doesn't get why we should store that.
[14:29]  f13: ajax: either persistant, or made on the fly, but in scm so that scm tools can be used to interact with it, for say path management or such.
[14:29]  jcollie: also, i think that we would want to make it possible to host a clone of an upstream repo
[14:29]  lennert: abadger1999: logic to handle what?
[14:30]  abadger1999: "losing information about where a change came from"
[14:30]  lennert: my point is, if you let people commit to the exploded tree directly, they'll just commit whatever
[14:30]  ajax: f13: that sounds like a finished statement, not a discussion.
[14:30]  lennert: and then all you can do is diff whatever last tarball you have against whatever most recent exploded tree you have, and shipping that diff
[14:30]  skvidal: lennert: yes, which is what mmcgrath said
[14:30]  skvidal: a mgmt mess
[14:31]  f13: ajax: no, no, there is still /lots/ of argument going on regarding this.
[14:31]  abadger1999: lennert: Not necessarily.  We've got people trained not to submit one uber patch a la Debian.
[14:31]  f13: ajax: whether or not it gives enough value to work around the pitfalls
[14:31]  lennert: skvidal: yeah
[14:31]  ajax: lennert: you seem to be assuming the scm can't handle branching intelligently, record unique commits...
[14:31]  abadger1999: lennert: We can do the same thing in the SCM.
[14:31]  ajax: git would have little trouble with this.
[14:31]  jcollie: i think that having a "pure" exploded tree package management repository is dead, everything else is up to discussuion
[14:31]  skvidal: ajax: no, he's assumg that people can't keep up with it
[14:31]  lennert: ajax: but how do you transport that info to .src.rpm ?
[14:32]  ajax: lennert: generate a patch series from the branch point to the tip, record both in the spec.
[14:32]  ajax: this is just stgit

[14:32]  lennert: ajax: problem with that is that you can't undo commits
[14:32]  lennert: well, stgit != git
[14:32]  abadger1999: lennert: Mercurial patch queues then.
[14:33]  lennert: assuming that all packages are in one repo (that is still up for discussion), rolling back would roll back other people's changes as well
[14:33]  lennert: abadger1999: or just plain quilt
[14:33]  abadger1999: Doesn't matter if it's built in or an addon.  the end product matters.
[14:33]  lennert: abadger1999: i.e. pretty much what we have now with .spec, Patch: and %patch
[14:33]  abadger1999: lennert: quilt is not the end all be all.  I've been using it since jwb packaged it for FE.
[14:33]  abadger1999: lennert: It needs to be integrated with the SCM.
[14:33]  ajax: yeah, automating %patch into the scm is really all i'm after.  i don't much care about having the whole project grafted in.
[14:33]  jcollie: ooh no i think that each package needs to be it's own repo
[14:34]  lennert: well, you can either put your quilt queue in svn or use stgit
[14:34]  lennert: jcollie: ideally you also want to track which version of which package is part of which distro
[14:34]  abadger1999: jcollie: At least, its own branch.
[14:34]  lennert: jcollie: you lose that info if you make each package a separate repo
[14:34]  f13: lennert: i think you're in the weeds.
[14:34]  abadger1999: Depending on terminology/scm/whatever.
[14:34]  lennert: jcollie: (i know, there are reasons for making each package live in its own repo as well)
[14:35]  lennert: f13: maybe
[14:35]  f13: lennert: you can easily make each 'distro branch' it's own repo for each package.
[14:35]  lennert: f13: that's backwards
[14:35]  f13: lennert: i did this with very little work when playing with dist-hg and dist-git.  Each package was a repo that has a "brnahes" file, each file listed what branches existed and would check out /those/ repos which were the content for those releases.
[14:35]  lennert: f13: so then how do i branch the distro?
[14:36]  f13: lennert: simply pull each packages + DISTRO/ repos
[14:36]  jcollie: lennert, you'd have to go into each repo and create the branch
[14:36]  chjunaidanwar has left ( (n=chatzill@203.101.183.120))
[14:36]  lennert: jcollie: that's just ugly
[14:36]  abadger1999: bzr multipull DISTRO/repos
[14:36]  lennert: jcollie: that's no better than what we have now
[14:37]  f13: lennert: what do you suggest?
[14:37]  f13: I'm very open to better workflow ideas.
[14:37]  lennert: well, why can't the set of packages just live in one repo?
[14:37]  jcollie: lennert, sure it would, at least each distro branch would be a real branch
[14:37]  lennert: then you could easily branch off f8 from devel, make an olpc branch, etc.
[14:37]  lennert: without having to add subdirectories to every package directory
[14:37]  lennert: just branch the entire repo
[14:38]  lennert: that makes it easier to add/delete packages as well
[14:38]  jcollie: i think the problem would be the size of the repo that you had to download
[14:38]  daMaestro has left ("Leaving" (n=jon@fedora/damaestro))
[14:38]  f13: lennert: for one thing, distributed scms would suck for that, as you'd have to suck down repodata for the entire thing just to work with one package.
[14:38]  abadger1999: lennert: Uhmmm... I'd be against that.
[14:38]  skvidal: jcollie: yes
[14:38]  f13: it'd be /huge/
[14:39]  lennert: f13: arguable, that would be a limitation in the scm
[14:39]  f13: we basically /have/ that with CVS, it's just that CVS offers you 'jump off' points.
[14:39]  skvidal: which is the other problem with exploded trees in the release
[14:39]  lennert: at least svn can check out/commit to subdirs
[14:39]  skvidal: s/release/scm/
[14:39]  skvidal: b/c it limits contributors w/slow connections
[14:39]  lennert: git doesn't handle that very easily right now but perhaps that could be added
[14:39]  jbowes: and git has submodules.
[14:39]  f13: lennert: yes, but cvs/svn leaves you completely tied to the upstream repo for everything, and having to be online.
[14:39]  abadger1999: lennert: Each branch can be a separate branch but only if it's downloadable separately by the VCS.
[14:40]  lennert: right
[14:40]  lennert: so instead of checking out /blah/fedora/packagename/branch
[14:40]  abadger1999: So for bzr and svn that is a valid implementation... for git and hg it is not.
[14:40]  lennert: we should think in terms of /blah/fedora/branch/packagename really, IMHO
[14:40]  jcollie: i was thinking of /blah/fedora/packagename
[14:40]  lennert: abadger1999: that functionality can be hacked in, though
[14:41]  lennert: jcollie: well, yes
[14:41]  abadger1999: lennert: By having a separate repo per branch
[14:41]  jcollie: then each release would have a git/ht branch
[14:41]  lennert: i fundamentally think that a branch should be a collection of packages, not that each package should have a separate repo branch
[14:41]  lennert: although technically in e.g. git it comes down to the same storage model
[14:42]  lennert: but i won't argue very strongly about it
[14:42]  ajax: it sort of depends whose life you're trying to make easy
[14:42]  jcollie: that still means that you have to download the specs/patches for every package (with git/hg at least)
[14:42]  jcollie: ajax, yeah
[14:42]  abadger1999: a branch should be a subunit of a package.  a repo should be a package.  a distro should be a collection of package-branches.
[14:43]  lennert: jcollie: so let's fix git then
[14:43]  ajax: as a packager, i often want to grab changes whole from one release train to another.  so i _do_ prefer the model of 'packages have branches' rather than the reverse.
[14:43]  f13: lennert: we can easily express the release as a directory of packages, thta's all in higher level 'tools' to get stuff.
[14:43]  lennert: jcollie: just that the current software doesn't easily let us do it doesn't mean we have to change the model to suit the currently available software
[14:43]  jcollie: lennert, git has submodules but it's a very new feature
[14:43]  ajax: but i suppose there's no deep reason you couldn't do both.  perforce has views for approximately this reason, and while i hate to say anything nice about perforce...
[14:44]  abadger1999: ajax
[14:44]  jcollie: i don't think that git 1.5.3 is in any fedora release yet, which is where proper support showed up
[14:44]  lennert: ajax: perhaps you could have both..
[14:44]  lennert: ajax: e.g. "give me a diff of the current directory between branch X and Y"
[14:44]  lennert: ajax: that doesn't mean that the branches should live under the package dir
[14:44]  ajax: "both" would mean the fundamental scm unit is package+release.  and you build views atop that, that give you either a package-centric or a distro-centric collection.
[14:45]  f13: lennert: likewise 'give me every package with branch FC-7' doesn't mean that we have to have our scm setup like scm/release/package
[14:45]  abadger1999: ajax: +1
[14:45]  ajax: essentially, bazillions of submodules, with magic view constructors.
[14:45]  lennert: ajax: define "scm unit"
[14:45]  f13: yep
[14:45]  lennert: ajax: you can have different paths leading to the same submodule, i.e. different ways of grouping them
[14:45]  ajax: right.
[14:45]  ajax: that.
[14:45]  lennert: f13: ack
[14:45]  ajax: we're agreeing.
[14:45]  lennert: ajax: ok
[14:46]  f13: progress!
[14:46]  lennert: hehe
[14:46]  rordway|m has joined the group chat (i=rordway@conference/oscon/x-61d319ee11b9488d)
[14:46]  skvidal: so does this mean if you
[14:46]  No5251 has left ("Kein Anschluss unter dieser Nummer!" (n=No5251@dslb-084-056-130-021.pools.arcor-ip.net))
[14:46]  skvidal: cd /scm/f7/
[14:46]  jcollie: lennert, maybe with CVS you can, but not really with git/hg... svn could kind of do it with svn:external
[14:47]  lennert: jcollie: why could you not have a commit to /blah/package to branch X auto-update the branch X super tree object or something like that?
[14:47]  lennert: jcollie: then people can still only just check out /blah/package, or check out the branch tree object, and get the same stuff
[14:48]  jcollie: i think that it unnecessarily complicates things
[14:48]  ajax: this is not, in fact, wildly different from how our cvs is set up now; i can _get_ just the devel branch of a given package.  it's just that we only construct one view with the SCM.
[14:48]  * skvidal is wondering how all this will play with local branches and following the upstream
[14:48]  abadger1999: Space
[14:48]  skvidal: ie: multiple maintainers maintaining it
[14:48]  jcollie: branching the entire distro can be easily scripted so that it's not much of a burden on rel-eng
[14:49]  lennert: ok, so from the package maintainer point of view, you tend to only care about one package
[14:49]  lennert: the 'branch the distro' use case isn't the popular one, it seems
[14:49]  lennert: right?
[14:49]  jcollie: yeah, i think we want to keep the maintainer's life as easy as possible
[14:49]  skvidal: yes
[14:49]  ajax: well.  i care about one package at a time, anyway.  i have lots of packages though.
[14:49]  skvidal: we have to keep this for the vast majority of contributors
[14:49]  skvidal: who are dealing with one package at a time
[14:49]  jcollie: since they are not likely to have high-speed access to the main repos
[14:49]  lennert: well, you could still create distro tree objects independently of what the maintainers do
[14:50]  * skvidal nods at ajax
[14:50]  lennert: so that people who care about branching the distro get to worry about that part
[14:50]  lennert: while the package maintainers worry about their part
[14:50]  skvidal: lennert: why not just step around that with makefile magic
[14:50]  abadger1999: Shoot, even one or two package-releases at a time in some cases.
[14:50]  skvidal: so I can cd to some dir and do:
[14:50]  skvidal: make grab-f7
[14:50]  ajax: yes, creating distro views is the part we're adding.  we already have the package view.
[14:50]  jcollie: rel-eng can run scripts on the same lan as the main repo so it's fast (enough) for them
[14:50]  f13: yeah I think we're agreed that we want distro views.
[14:51]  ajax: the only other feature i'm interested in is automating %patch
[14:51]  ajax: and i think i can get that with quilt or stgit or whatever
[14:51]  lennert: are there cases where you are interested in a partial order on patches going to different packages?
[14:51]  skvidal: ajax: automating how?
[14:51]  skvidal: ajax: we still need a patch in the srpm
[14:51]  lennert: e.g. when you are applying dependent patches to different packages
[14:51]  skvidal: ajax: and we need it to not connect out to the world when building
[14:51]  jcollie: ajax, what if we had some makefile magic to pull a pactch out of a separate repository?
[14:52]  ajax: skvidal: right, so, say i'm working on the X server, which is in git.
[14:52]  ajax: and that we use t for the fedora scm
[14:52]  * lennert likes git
[14:53]  skvidal: ajax: ok
[14:53]  ajax: the srpm tarball is now a complete repo, that contains both the master branch (upstream release), and a fedora branch that i'm allowed to play with for patch generation.
[14:54]  ajax: i do my commits on the fedora branch
[14:54]  f13: hrm.
[14:54]  tibbs has left ("Konversation terminated!" (n=tibbs@fedora/tibbs))
[14:54]  ajax: and i get git-rebase if i want it, but that's later.
[14:54]  jcollie: what if we had a way to specify a git://url and two commit ids, and the makefile would automagically pull down a copy of the x.org git tree, extract the patches (git format-patch X..Y) and fix up the spec file and add the patch files to the package repo?
[14:54]  f13: ajax: I think that breaks directive #1, which is srpms need to be prestine tarball + our junk, the tarball must be untouhced if at all possible for verification.
[14:55]  ajax: now, at %prep time, i unpack the tarball (which gives me master), and generate the patch trail between master and fedora branch.
[14:55]  f13: ajax: but maybe you can still get there if you have prestine tarball, and you overlay it with .git/
[14:55]  skvidal: ajax: so your tarball would really be your own fork of X, for example
[14:55]  ajax: f13: that's a bogus requirement anyway.  you have sha1sums for master.
[14:55]  f13: from a second tarball.
[14:55]  ajax: in fact, that's how git revs are specified, period.
[14:55]  ajax: it's exactly as verifiable as before.
[14:55]  jcollie: interesting thought... if you have a git tree object id, git guarantees that you have the same tree as someone else
[14:55]  skvidal: ajax: verifable to the source: path?
[14:56]  f13: ajax: can you explain a bit more on that, as there was violent reaction to not having prestine tarballs here.
[14:56]  skvidal: ajax: how do I as a random user figure out which tarball you started from?
[14:56]  f13: ajax: pretend we're newbies that don't know git.
[14:56]  skvidal: let's say I can find the srpm
[14:56]  ajax: okay
[14:56]  jbowes: violent reaction to not having pristine sources. it doesn't have to be from a tarball, so long as it is verifiable
[14:56]  ajax: please, one at a time.
[14:56]  skvidal: sorry

[14:57]  f13: ajax: do skvidal first.
[14:57]  * lennert kind of sees ajax's point
[14:57]  skvidal: f13: eww
[14:57]  marek has joined the group chat (n=pyxel@fedora/pyxel)
[14:57]  * jcollie reminds everyone that we are about to bump into fedora-infrastructure's meeting time...
[14:57]  skvidal: ajax: It sounds like you're suggesting that fedora's tarball become the intermediary for verification
[14:57]  mmcgrath:
[14:58]  ajax: skvidal: yes, verifiable to the source path.  xserver/.git/config contains [remote "origin"] , which contains the url I fetched from.  anyone who wants to verify that my source is the same as what came from upstream, can just clone from there and compare commit IDs.
[14:58]  mmcgrath: jcollie: you want to wrap this meeting up?
[14:58]  abadger1999: jbowes: verifiable for who from where, though?  We can keep a sha1sum similar to how we keep a lookaside but that doesn't help the enduser.
[14:58]  skvidal: mmcgrath: /join #fedora-meeting2
[14:58]  lennert: ajax: doesn't that require that upstream use git?
[14:58]  vpv has joined the group chat (i=vpv@kapsi.fi)
[14:58]  jcollie: yeah, since we e starting to get a little outside the agenda that we set up
[14:59]  marek:
[14:59]  ajax: commit ids in git are a sha1 sum over (essentially) all the change information that went into the tree, from creation to that point in time.  so the odds of collision on any single hash are astonomically small, because you have to _also_ forge all the previous hashes.
[14:59]  Sopwith has joined the group chat (n=elee@216.27.62.2)

[14:59]  mmcgrath: jcollie: its nice to see people show up and talk about it though.
[14:59]  jcollie: yeah
[14:59]  ajax: lennert: yeah, but that's just a matter of time
[14:59]  lennert: ajax: you could indeed just generate a patch set between master and fedora branch and stick that into the srpm
[15:00]  skvidal: ajax: umm, no
[15:00]  lennert: ajax: hehe
[15:00]  skvidal: ajax: anything that relies on upstream doing anything consistent is a non-starter
[15:00]  abadger1999: ajax: No one will be using git in 20 years
[15:00]  ajax: skvidal: you don't understand crypto.
[15:00]  f13: yeah I veto that idea too, that it relies up on upstream being git.
[15:00]  ajax: skvidal: all upstream objects have to be consistent because the tool enforces it.
[15:01]  skvidal: ajax: what are you talking about?
[15:01]  lennert: instead of certifying the upstream commit object you derive from, you'd certify the tree object
[15:01]  f13: couldn't we accomplish ajax's needs by having the prestine tarball, and a second tarball that is the .git/ data thta gets overlayed upon extraction?
[15:01]  ajax: f13: no, it relies on upstream using a crypto-enabled scm.  hg and monotone and bzr have the same property.
[15:01]  jcollie: ok, i hate to cut off discussion, but we need to turn the channel over to mmcgrath / fedora-infrastructure... we do have a #fedora-scm channel though for some informal discussion
[15:01]  ajax: and if it doesn't, then you just suffer and make SRPMs the legacy way.
[15:01]  lennert: #fedora-scm sounds good to me
[15:01]  ajax: yay, another channel
[15:02]  jcollie: i think that the summary for the meeting is that verification of upstream sources is the ultimate blocker for changing the way we do business
[15:03]  lennert: jcollie: ack
[15:03]  jcollie: i'll post the log and that summary to -devel in a little while
[15:03]  mmcgrath: jcollie: well....  also while making sure we don't creata feature that only 5% of our users will use while the other 95% have to do more work
[15:03]  jcollie: right
[15:03]  mmcgrath: just making sure we don't lose sight of that

[15:04]  mmcgrath: jcollie: all done?  Should I change the topic?
[15:04]  jcollie: yep go ahead... looks like #fedora-scm took the discussion over
[15:04]  mmcgrath: excellent.