Dist Git Branch Proposal

This page is to track a proposal on changing how we have our branches structured currently in dist-git

Email Proposal

I'm working on fixing a few long standing bugs in fedpkg that have to do
with our branch structure
(https://bugzilla.redhat.com/show_bug.cgi?id=619979 and
https://bugzilla.redhat.com/show_bug.cgi?id=622592).

This has me examining our branch structure again and trying to remember
why exactly I chose it (obviously I did a poor job of documenting that...).

The original thought was to have top level branches that are named after
distribution releases, eg "f14", "f15", "el5".  Then we would force
branches of those branches use a naming structure of "f14/topic".  The
reason was so that our tooling could look at the name of the branch and
easily work back to the "f14" part.  This would work even if it was
"f14/user/fred/topic/mybranch"  or other such craziness.  When I went to
test this, I realized that git won't allow you to have both "f14" and
"f14/topic" as branches, because of the way the git metadata works on
the filesystem.  When I encountered this, I made "f14/master" become the
top level branch, and then "f14/somethingelse" could coexist.
Unfortunately I also wanted to keep things easy for users and tried to
maintain tooling that would allow you to just say "f14".  This didn't
get enough real world testing and in hindsight was a bad idea.  Things
go wrong quickly in git if your local branch name doesn't match the
remote branch name.

When thinking about the above, and the two bugs I'm working on, I
realized that we don't have any real strong need to be using "/" as a
delineator.  It makes some code easier, but makes other things more
complex and difficult.  So what if we changed it?

What I'm thinking about now is switching from / to - as a delineator.
This would improve a couple things.  First, we could achieve upstream
top level branch names that are short and simple: "f14", "f15",
"master".  We can have branches that build upon those names:
"f14-rebase", "f15-cve223", "f15-user-jkeating-private".  We could keep
the simple fedpkg tooling that allows users to just type "f14" and the
like to reference a branch, and now the local branch will match the name
of the remote branch.

As for the first two bugs I mentioned, it doesn't directly help them.
However I would feel better about telling people that their local
branches must follow a naming scheme of <release>-<something> and then
we could easily guess what release the local branch is for if it isn't
tracking a remote branch.  However the bug about what to do if there are
no remote branches is really not touched by any of this, it just got me
thinking about branches :)

What kind of user impact would this have?

My hope is that the impact would be minimal.  Git allows branch renames,
and can successfully rename "f14/master" to "f14".  All the history is
renamed.  We should be able to do this without an outage of the git server.

The ACL system will need a slight tweaking, and a regeneration of the
ACLs but that is fairly quick and minor to accomplish.

However there will be some issues client side.  We will not be able to
make use of git's symbolic-ref feature of "aliasing" a branch.  We
cannot make "f14/master" an alias for "f14", again because of the
filesystem layout issues.  These issues rear their head once again when
a client does a pull of an already checked out repo that had branches.
Because there was already a f14/master, when the client sees a new
branch just named "f14" it will fail to create it.  Git has a command
that will fix this "git remote prune origin".  That will remove the
local reference to f14/master and a subsequent pull sees the creation of
the "f14" branch happen successfully.  However, if a user had a local
branch of f14 or f14/master they will be left with mismatched
.git/config entries.  In this case it's easiest to delete the local
branch (git branch -d f14) and check it out again.  If there are changes
on the branch one can fix the config to point it to the right upstream
location.

Also we would need to get a new fedpkg into the hands of all the
developers that handles the new branchnames.  We could do a build that
handles both the oldnames and the new and have it out and available for
a reasonable period of time before we make the switch.

So, some pain, for some pretty good gain.  This time around I can setup
pkgs.stg with this branch configuration and builds of fedpkg to use it
for a more through testing before rolling it into production.

So please, tell me what you think!

Impact

This change will impact a few things.

Git Server

Various changes would happen on the git server

The repos

The git server is where we would process all the rename jobs. A script needs to be written in order to discover and rename the branches appropriately.

Some care will have to be taken when there exists release branches beyond f14/master. The other branches will need to get renamed to f14-<branch> before finally renaming f14/master to f14.

The ACL system

The ACL system will need to be tweaked to remove the trailing "/" from branch path names in the ACL files.

We can also modify the ACL system to detect push attempts that include a path with "<branch>/<something>" and instead of generically erroring out to the user we can print a link to a wiki page explaining the migration and what the end user must do.

Branch Creation Scripts

There are a couple branch creation scripts that will need to be updated to reflect no longer having "/master" in branch names.

fedpkg

The branch naming comes up surprisingly few times in fedpkg code. It is fairly trivial to go from "<foo>/master" to just "<foo>". The harder part will be putting in code that handles both scenarios at the same time, so that we can roll the new fedpkg into place well ahead of the git server work.

Testing

Currently we are using pkgs01.stg.phx2.fedoraproject.org as a testing environment for changing the various server side elements listed above with client side changes. At this time it is not appropriate for public testing, nor do we have a fedpkg build for client testing.

The test plan would be to sync pkgs.stg.fedoraprojec.org with some of the production pkgs.fedoraproject.org and then convert all the branches and scripts on pkgs.stg. Then provide a build of fedpkg that is configured to use this host and configured to use the new branch names to interested testers. Feedback will be processed and things potentially tweaked until we feel that the setup is sufficient.

Timeline

We would like to complete the migration prior to or just after FUDCon Tempe 2011, which means late January 2011. Date subject to change as this has not been proposed to FESCo yet.

Here is a quick todo list:

• Investigate changes requires a bit more using pkgs.stg -- IN PROGRESS
• Propose change to FESCo -- DONE and APPROVED
• Setup pkgs.stg for widespread testing -- DONE
• Publish a build of fedpkg appropriate for testing -- DONE
• Apply ACL patch to warn users on push attempt to old branch path -- DONE (locally, not in a package)
• Write client side script to update clone metadata -- DONE
• Solicit testing and feedback -- Got some testing going
• Prepare production build of fedpkg
• Schedule time for production changeover of pkgs.fedoraproject.org
• Execute changeover
• Prepare and publish another build of fedpkg that removes code to deal with old branch style

Proposal Owner

Jesse Keating