Short description
GitLab is an open source MIT licenced git repository management application. It is built on Ruby on Rails and is used by many companies as their internal git management repository. The reason it gained so much popularity is that it bares a strong resemblance to github's looks and feels. It is a project with great potential, under heavy development with a release cycle every month. That "release early, release often" strategy makes it possible to apply bug fixes quite regularly and add new features.
This proposal is aiming to package GitLab and all its dependencies for Fedora and EPEL.
What are the benefits
- Supporting more ruby packages would make Fedora the appropriate distribution for ruby development.
- Fedora is one of the biggest open source communities (if not the biggest) so far, with a very strong community base. If GitLab were to be supported officially, it would be the first distribution to do so and could be one of their supported platforms apart from Debian/Ubuntu.
- I have seen many installations on CentOS (forum, issue tracker, blogs), there is demand. rpm packages means "One click install" for Fedora, Red Hat, CentOS and other rpm based distributions. Huge profit, as sysadmins will not have to worry about the manual installation process.
- When the packaging is done GitLab could be deployed as an extra git service for fedorahosted.org. I expect many people to switch their git hosting from the bare gitweb to GitLab. It will enhance collaboration, encourage participation and make coding more fun as it will be an interactive process.
Relevant expertise
Hands on experience
- I have been using/testing GitLab since version 2.0.
- I actively participate in the forum talk, I contribute upstream and I am in direct contact with its developers.
- I have deployed it over 20 times, mostly on Debian, so I am familar with the process.
I have spent several hours contributing not only upstream but I also wrote a wiki article for Archlinux as well as an installation script and recently I gave a small talk in greek about GitLab during a hackfest.
How I intend to implement my proposal
Gem hunting
The first step would be to compile a list of the gems GitLab is using and compare it against the ones already packaged in order to find those missing from the repos.
GitLab gems
Using the Gemfile.lock is a good starting point to compile the needed list. In it there are all the gems GitLab depends on. Many of them are duplicate, so using this little python script leaves us with about 200 gems. Of those 200, most are downloaded from rubygems.org, some are pulled from git repositories, whereas some are patched by GitLab.
However, those gems are not final, as the script doesn't take into account (yet) the development packages, which we don't necessarily need in order to have a working GitLab instance. On the other hand, some of them are needed by the test suite, so one must be careful of what to exclude. I will rework on that script and plan to write it in ruby as well.
Fedora gems
Next step is to find which of the gems above are already packaged for Fedora. I have crafted a little bash script, which searches the repositories of the current Fedora release (no testing/rawhide enabled) for gems. We come up with 387 total gems of which 93 are GitLab's dependencies. In particular:
Gitlab uses 199 gems. Fedora has packaged 387 gems. There are 93 common gems. There should be packaged 106 gems.
Those 106 gems are not final, as there are also others which are not yet in Fedora, but they are packaged by their upstream. Also, there are many gems pending review in bugzilla.
Unfortunately, there is a caveat which needs to be addressed and that is version mismatch. More about this in #Caveats_and_possible_problems_I_may_encounter.
Packaging for Fedora/EPEL
As stated in the wiki, there are three basic categories of ruby packages: ruby gems, non-gem ruby packages, and applications written in ruby. Fortunately, all dependencies are ruby gems which makes it easier to package (details below). GitLab itself is a rails app which needs extra care.
To make matters easier, new packages should go only into Rawhide. Once Rawhide turns into Fxx, GitLab should be in some fixed stable version in the rest of Fxx life cycle. Once the packaging for Fedora is done, next step is EPEL6, either by editing the existing spec files and using conditions or keep a different version of .spec in each branch. There exists the katello koji which will be a good source of already packaged gems. They have many packaged in EPEL6 as well.
Gems
Gems contain three components:
- Code (including tests and supporting utilities)
- Documentation
- gemspec
They share the same structure so it should be fairly easier to package them.
foo/ ├── bin/ │ └── foo ├── lib/ │ └── foo.rb ├── test/ │ └── test_foo.rb ├── README ├── Rakefile └── foo.gemspec
(Taken from http://guides.rubygems.org/what-is-a-gem/)
Fortunately, there is a tool named gem2rpm which parses the info inside the gemspec and creates a spec file. It also has this nice feature of creating a template rather than a spec file. This facilitates the update process and makes it easier to update the RPM when a new version of the Gem becomes available. This tool will be heavily used for the gem packaging.
On the other hand, there is no way to completely automate the conversion, so some fields will be filled by hand. According to rubygems, the required gemspec attributes are:
- files
- name
- platform=
- require_paths
- rubygems_version
- summary
- version
whereas attributes such as the license or the changelog are not mandatory and we will have to add it ourselves..
GitLab
Once gems are done, it is time to package GitLab itself. There is a new version every month, so Rawhide will always have the latest one. At this present time, GitLab is installed in the home directory of user 'git', but since the RPM packages must obey FHS rules it should be installed into %{_datadir}.
Service files
systemd
For newer versions of Fedora we will need systemd service files. That won't be a difficult task since there are already some unofficial services floating around. All they need is some refinement.
rc.d
An rc.d service file is already provided by upstream for CentOS.
MariaDB support
Since MariaDB will replace MySQL, GitLab must be tested against MariaDB. At this point there should not be any incompatibility. Upstream doesn't yet support it so this should be discussed with them.
Caveats and possible problems I may encounter
Version mismatch
One thing to consider before packaging, is the version mismatch between gems used by GitLab and those packaged/to be packaged for Fedora. Already, GitLab uses about 10 runtime packages pinned to a lower version than upstream. Although it is always a good idea to package the latest version, we might end up packaging a lower version in order to get GitLab to work. If the package is already in Fedora and we need an older version it would be better to test GitLab against the version in Fedora.
On the other hand, of the Ruby gems that are packaged, many of them are the wrong version, and for some of the Ruby gems that GitLab requires, it uses git snapshots of non-released versions, or versions that have been forked/patched by GitLab developers.
So, the first step prior to packaging, will be to compile a list with gems and their corresponding version. Maybe a table like this would be useful:
Gem name | Upstream version | GitLab version | Fedora version
isfedoraruby and gymnasium are two good tools to find out about version mismatch.
Test suite on koji
Test suite requires a running MySQL server instance, which means that the test suite can't run inside Koji.
A rough timeline
There are almost 100 gems to be packaged. The concept is to use 2 months at most to finish packaging and then begin testing the whole application on a production server. I estimate to package roughly 2 gems/day including their bugzilla submission.
27 May - 17 June
During the bonding period I intend to thoroughly read the packaging documentation and familiarize with the tools that will be used throughout the project:
- gem2rpm
- rpm
- mock
- fedpkg
- koji
- etc
A preliminary gem packaging process with hands-on use of the mentioned tools is also scheduled.
18 June - 29 July
18-30 June
Compile the gem list and sort according to the following table:
Gem name | Upstream version | GitLab version | Fedora version
That will make matters easier to work afterwards. I will also make sure to include the dependencies of each gem in order to prioritize which gems to build first to avoid conflicts.
Reach any third party that are packaging for Fedora/EPEL (eg. Kastello people) and see if any of GitLab's gems are already packaged. Kastello has many packages ready for EPEL6, so it will be helpful later when packaging for EPEL begins.
1-29 July
- Start packaging and submit to bugzilla.
- Test that the packages work in Rawhide.
- Make any corrections needed and repeat steps.
After those 4 weeks, the packaging process should be around 60-70%.
29 July - 2 August
3 August - 16 September: pencils down
23 September - 27 September: final evaluation
27 September: Submit to google
Longterm commitment
It goes without saying that without longterm support the whole packaging process would have no meaning.
Team of maintainers
Maintaining almost 100 packages by oneself is quite a task, so there must be formed some sort of a team willing to take over some packages and help with the maintenance.
Nice to have
Puppet/Chef
Easy deployment on multiple servers is something a sysadmin would like to have. As far as GitLab is concerned, there are two puppet modules actively developed (unofficially), whereas there is a suggestion by a GitLab developer for an officially supported Cookbook.
Other details
GitLab supports Ruby version >= 1.9.3 and the new 2.0, so there is no problem regarding that.