GSOC 2014/Student Application Axilleas/Isitfedoraruby

isitfedoraruby.com is a web-application built with Ruby on Rails, that keeps track of the Fedora/Ruby integration, especially gem -> rpm conversion.

The project was started on 2012 as a Google Summer Of Code project by User:Zuhao and later were added some new features during the Google Code In program. This is my idea of enhancing it from a packager's point of view.

An overview of your proposal

isitfedoraruby is a nice project that never got the attention it deserved. I believe this is because it serves merely as a portal of the gems packaged for Fedora, showing their statuses and some nice statistics and graphs.

Being a packager myself for almost a year now, I am familiar with the difficulties one has to overcome in order to package a gem. This idea aims to facilitate the packaging process by providing some new tools and promote the site among Fedora ruby packagers and the rest of the ruby community.

Some of the ideas include:

• refactor the base code where needed
• write rspec tests
• enhance the gemfile_tool
• implement a feature of gem/app packaging progress bar
• enhance the dependency checker
• create a preview page with stats of a gem not yet packaged
• use polisher gem where needed
• write documentation and how-to guides of rubygem packaging
• UI - UX changes
• make badges like travis-ci/codeclimate/gemnasium/etc so that each project could show that they have the library packaged for Fedora

Just a note that implementing all the items above, might take more than three months as GSoC officially lasts. During the bonding period (April 21 - May 18) I will decide which to implement first and which will follow after the program ends.

The need you believe it fulfills

These enhancements will hopefully promote the site usage among Fedora ruby packagers and attract new ones.

For new packagers

Based on my experience as a new rubygem packager a year ago, I have compiled some useful information for the packaging process that would help new users to engage in ruby packaging much quicker than I did. You can read a draft here. This will be accessed through a special docs page on isitfedoraruby.com and could be edited with pull requests in the github repo.

I also plan to document the workflow a packager follows to build/update a gem, the tools needed to achieve this and the ones that would make the process easier, e.g. using the polisher gem and its binaries.

Of course documentation is not the only important thing, new packagers could also use the advanced features I describe below.

For existing packagers

The new enhancement that I have in mind for the gemfile_tool will help packagers that want to package an application with many dependencies. For example, Gitorious and GitLab are two big rails projects with over 100 dependencies each. The way gemfile_tool is now implemented isn't of much use. Using the polisher gem we can enhance it to depict the dependencies in a nice format (eg. table), export to json, display only the runtime dependencies, etc.

Any relevant experience you have

To be exact, it's been a year since I got involved in learning the ruby language and the rails framework. Since then I have deployed numerous rails applications, the most complex being GitLab which was included in my previous GSoC proposal. I have written quite a few tools/scripts in ruby (mostly for personal use) and also some simple rails apps.

isitfedoraruby is the first big application that I will be working on. Apart from ruby/rails, I have a solid knowledge of linux since I use it everyday for the last 8 years.

How you intend to implement your proposal

Refactor basecode where needed

Zuhao has been maintaining the app for quite some time now, with recent changes the port to rails4. That said, an important amount of work is already done. From my side I will try to find if the existing code needs any refinement.

Write tests

The current test suite is almost non-existent. I will use:

• rspec for models/controllers
• factory_girl for feeding tests
• capybara to simulate users' interaction with the app
• selenium-webdriver to test javascript interactions

For views I will use feature specs. Existing minitest tests will be replaced with rspec ones.

As a first step, I will make a list of things that need testing, as well as reporting some existing bugs that will be added to the tests to avoid future repetition.

Enhance gemfile_tool

gemfile_tool has the ability to parse a given Gemfile{,.lock} and tell you whether the gems found are already packaged and in what Fedora versions. In its current implementation it uses a hacky way to calculate the gems. Also, it has the drawback that it uses cached versions taken from Fedora's repos once a day through a cron job.

On the other hand, using the polisher gem we can query a bunch of services (koji,buzilla,rubygems.org) on the fly and learn about the status of the gems at the given time.

As an enhancement, one will be able to see a table like the one I keep track of GitLab's dependencies.

Show the packaging progress of a rails app or a gem

A separate page that shows the packaging progress of a gem will be provided.

Scenario

The packager submits a Pull Request on the issue tracker providing a yml file. When the patch gets merged and deployed to production, a new page will be created depicting the packaging progress of this certain gem.

Implementation

The yml file will contain information of the gem name as per the review request and the bugzilla issue number. Each time the gem page gets loaded, it will query koji for rawhide builds and bugzilla for any Blocks issues. If we have those two values we can calculate how many dependents gems there are not yet or already packaged.

A nice progress bar with the percentage of gem packaging completion will be provided as well. For each gem that gets in rawhide, the progress bar gets a little further. There can also be a number of other cool info in the page, like links to bugzilla requests so that someone could lend a hand, etc.

I might "steal" the idea from GitLab on how they use milestones :)

Future enhancement

At first I will focus on implementing this for a single gem but later it can be extended to track the progress of a rails app.

Enhance dependency checker

The current representation of a gem's dependencies is in a tree format, not very handy if you want to extract information.

Influenced by https://www.gemlou.pe, I plan to implement a similar representation and show the dependencies in a more elaborate way, for example in a table format like:

 gem1
 |- gem2
 |- gem3
     |- gem4

This feature will also be available for gems not yet packaged, with an extra addition: one could find out which gem should start packaging first, that is this that has no dependencies or its dependencies are already packaged. For example consider a gem with three dependencies (children) and each of them has some children as well.

 gem1
 |
 |- gem2 (packaged)
     |
     |- gem5 (implies packaged)
     |
 |- gem3 (not packaged)
     |
     |- gem6 (packaged)
     |
     |- gem7 (not packaged)
     |
 |- gem4 (not packaged)
     |      
     |- gem8 (not packaged)

Implementing a breadth-first search algorithm we can move along and see whether the children nodes are packaged. Starting with gem2, since it is packaged, that implies that its children are also packaged so we move on to the next gem3. Following this process we can tag which gems are free of dependencies, so that they could be packaged first and build up the chain.

Special pages

Pages that would help the user experience are:

• list of gems pending review
• list of gems already assigned but stagnated for too long
• list of gems pending review and marked as NEEDSPONSOR

All these can be queried from bugzilla through the ruby-bugzilla gem.

RSS feeds

To make it easy to follow up one's packages, I will implement rss feeds with categories:

• not up-to-date
• pending review
• etc

UI-UX changes

The new revamped version of isitfedoraruby uses the bootstrap3 framework which gives us the power to easily use certain css features and provide better visual output. The plan is to make sure that the site is responsive and the user experience maximized.

Badges support

One cool idea is to provide badges like travis-ci, gemnasium, coveralls, etc., hooked to isitfedoraruby and show the existence of a Fedora packaged gem.

I'm note really sure how that would be implemented, but it's a nice idea to promote the site and the work done by Fedora rubygem packagers.

A rough timeline for your progress

March 27 - April 27

The first month I will study the codebase, report any existing bugs and get in touch with the current maintainers and straighten out any queries I might have. During this time, I will also decide which parts might need a cleanup.

This will be a good time to ask the Fedora community via the ruby-sig ML, which features they think need more attention and brainstorm any other possible ideas that I haven't come up with.

April 28 - May 19

After the discussion with fellow rubyists, I will decide the features that will eventually be implemented.

The next 4-5 weeks will be dedicated to writing tests, so before the official coding date begins, I will make a comprehensive list of what needs testing. This will be a good opportunity to visualize the user cases as well.

May 19 - June 27

May 19 - June 15 (4 weeks)

As stated earlier, the first 4-5 weeks I will be writing the test suite which will be divided to the following steps:

Unit testing

• setup rspec
• write models specs (test instance methods and classes)
• write controller specs
• generate test data with factory_girl

Integration testing

• use capybara gem for feature specs

June 16 - June 23 (1 week)

The final week will be dedicated to make sure every test passes and speed testing is of normal levels.

June 24 - June 27

Midterm evaluation. Wrap up anything done so far and write documentation.

June 27 - August 18

The following 8 weeks will be when new features will be added. Assuming that each feature might take 2 weeks here is a rough timeline.

June 27 - July 13 (2 weeks)

Two weeks to implement the dependency checker feature should be enough.

July 14 - July 27 (2 weeks)

Another two weeks to implement the gemfile_tool feature. Since I will be relying on polisher gem, if by then it doesn't support the needed functionality, I will work on polisher as well.

July 28 - August 10 (3 weeks)

Last 3 weeks will be for the preview gem info and progress bar features. I decided to work on them at the same time, since the progress bar feature kind of relies on being able to see the information of a gem not yet packaged for Fedora.

August 11 - August 18 (1 week)

Final week to sum up what is done until that time, document what needs documentation, test that everything works.

Nice to have

Here are some leftover thoughts I had that could be added at a later date.

User logic (fedauth support)

As much as nice it is to have users interacting with your application, this would add an amount of work to be done as well as it brings on a whole layer of security issues.

Package application and include to apps.fp.o

It's well known (at least to me :p) that packaging and maintaining a rails app requires quite an effort. If we reach a point where isitfedoraruby is used by many ruby packagers and other people in general, it would make sense to deploy it in Fedora infrastructure.

Use by other distros

I have stated many times the use of polisher in the features I propose. polisher is in fact written to support other distributions as well, although not all features are implemented. At a later point, we could add support for other distributions except Fedora. But that's a thought :)

Any other details you feel we should consider

Some features I have added to the proposal might not be implemented during GSoC. That doesn't mean that they won't get implemented at all since I will continue to support the application with bug fixes and enhancements.

Have you communicated with a potential mentor? If so, who?

Yes, User:vondruch