GSOC 2013/Student Application Axilleas/Gitlab(463)

Short description

GitLab is an open source MIT licenced git repository management application. It is built on Ruby on Rails and is used by many companies as their internal git management repository. The reason it gained so much popularity is that it bares a strong resemblance to github's looks and feels. It is a project with great potential, under heavy development with a release cycle every month. That "release early, release often" strategy makes it possible to apply bug fixes quite regularly and add new features.

This proposal is aiming to package GitLab and all its dependencies for Fedora and EPEL.

What are the benefits

• Supporting more ruby packages would make Fedora the appropriate distribution for ruby development.

• Fedora is one of the biggest open source communities (if not the biggest) so far, with a very strong community base. If GitLab were to be supported officially, it would gain much more reputation and probably set up as the de facto git management system for other organizations that want control over their personal git repositories.

• Seen many installations on CentOS (forum, issue tracker, blogs), there is demand. rpm packages means "One click install" for Fedora, Red Hat, CentOS and other rpm based distributions. Huge profit, as sysadmins will not have to worry about the manual installation process.

• When the packaging is done GitLab could be deployed as an extra git service for fedorahosted.org. I expect many people to switch their git hosting from the bare gitweb to GitLab. It will enhance collaboration, encourage participation and make coding more fun as it will be an interactive process.

Relevant expertise

Hands on experience

• I have been using/testing GitLab since version 2.0.

• I actively participate in the forum talk, I contribute upstream and I am in direct contact with its developers.

• I have deployed it over 20 times, mostly on Debian, so I am familar with the process.

Other related activities

I have spent several hours contributing not only upstream but I also wrote a wiki article for Archlinux as well as an installation script and recently I gave a small talk in greek about GitLab during a hackfest.

How I intend to implement my proposal

Gem hunting

The first step would be to compile a list of the gems GitLab is using and compare it against the ones already packaged in order to find those missing from the repos.

GitLab gems

Using the Gemfile.lock is a good starting point to compile the needed list. In it there are all the gems GitLab depends on. Many of them are duplicate, so using this little python script leaves us with about 200 gems. Of those 200, most are downloaded from rubygems.org, some are pulled from git repositories, whereas some are patched by GitLab.

However, those gems are not final, as the script doesn't take into account (yet) the development packages, which we don't necessarily need in order to have a working GitLab instance. On the other hand, some of them are needed by the test suite, so one must be careful of what to exclude. I will rework on that script and plan to write it in ruby as well.

Fedora gems

Next step is to find which of the gems above are already packaged for Fedora. I have crafted a little bash script, which searches the repositories of the current Fedora release (no testing/rawhide enabled) for gems. We come up with 387 total gems of which 93 are GitLab's dependencies. In particular:

 Gitlab uses 199 gems.
 Fedora has packaged 387 gems.
 There are 93 common gems.
 There should be packaged 106 gems.

Those 106 gems are not final, as there are also others which are not yet in Fedora, but they are packaged by their upstream. Also, there are many gems pending review in bugzilla.

Unfortunately, there is a caveat which needs to be addressed and that is version mismatch. More about this in #Caveats_and_possible_problems_I_may_encounter.

Packaging for Fedora/EPEL

As stated in the wiki, there are three basic categories of ruby packages: ruby gems, non-gem ruby packages, and applications written in ruby. Fortunately, all dependencies are ruby gems which makes it easier to package (details below). GitLab itself is a rails app which needs extra care.

To make matters easier, new packages should go only into Rawhide. Once Rawhide turns into Fxx, GitLab should be in some fixed stable version in the rest of Fxx life cycle. Once the packaging for Fedora is done, next step is EPEL6, either by editing the existing spec files and using conditions or keep a different version of .spec in each branch. There exists the katello koji which will be a good source of already packaged gems. They have many packaged in EPEL6 as well.

Gems

Gems contain three components:

• Code (including tests and supporting utilities)
• Documentation
• gemspec

They share the same structure so it should be fairly easier to package them.

foo/
├── bin/
│   └── foo
├── lib/
│   └── foo.rb
├── test/
│   └── test_foo.rb
├── README
├── Rakefile
└── foo.gemspec

(Taken from http://guides.rubygems.org/what-is-a-gem/)

Fortunately, there is a tool named gem2rpm which parses the info inside the gemspec and creates a spec file. It also has this nice feature of creating a template rather than a spec file. This facilitates the update process and makes it easier to update the RPM when a new version of the Gem becomes available. This tool will be heavily used for the gem packaging.

On the other hand, there is no way to completely automate the conversion, so some fields will be filled by hand. According to rubygems, the required gemspec attributes are:

• files
• name
• platform=
• require_paths
• rubygems_version
• summary
• version

whereas attributes such as the license or the changelog are not mandatory and we will have to add it ourselves..

GitLab

Once gems are done, it is time to package GitLab itself. There is a new version every month, so Rawhide will always have the latest one. At this present time, GitLab is installed in the home directory of user 'git', but since the RPM packages must obey FHS rules it should be installed into %{_datadir}.

Service files

systemd

For newer versions of Fedora we will need systemd service files. That won't be a difficult task since there are already some unofficial services floating around. All they need is some refinement.

rc.d

An rc.d service file is already provided by upstream for CentOS.

MariaDB support

Since MariaDB will replace MySQL, GitLab must be tested against MariaDB. At this point there should not be any incompatibility. Upstream doesn't yet support it so this should be discussed with them.

Caveats and possible problems I may encounter

Version mismatch

One thing to consider before packaging, is the version mismatch between gems used by GitLab and those packaged/to be packaged for Fedora. Already, GitLab uses about 10 runtime packages pinned to a lower version than upstream. Although it is always a good idea to package the latest version, we might end up packaging a lower version in order to get GitLab to work. If the package is already in Fedora and we need an older version it would be better to test GitLab against the version in Fedora.

On the other hand, of the Ruby gems that are packaged, many of them are the wrong version, and for some of the Ruby gems that GitLab requires, it uses git snapshots of non-released versions, or versions that have been forked/patched by GitLab developers.

So, the first step prior to packaging, will be to compile a list with gems and their corresponding version. Maybe a table like this would be useful:

 Gem name | Upstream version | GitLab version | Fedora version

isfedoraruby and gymnasium are two good tools to find out about version mismatch.

Test suite on koji

Test suite requires a running MySQL server instance, which means that the test suite can't run inside Koji.

Longterm commitment

It goes without saying that without longterm support the whole packaging process would have no meaning.

Team of maintainers

Maintaining almost 100 packages by oneself is quite a task, so there must be formed some sort of a team willing to take over some packages and help with the maintenance.

Other details

ruby version. supports >= 1.9.3 , 2.0 rails 4, no support EPEL6

Until it's packaged and up to Fedora guidelines, we aren't likely to look at deploying it.

Once the packaging hurdle is overcome, yeah, we would want a team to help manage it and deploy it and keep it running.