Project Title : Fresque
|
Contact Information |
|
|
About Me |
|
|
Q&A |
I have been using Fedora for past five years and I am fond of its user friendly interface and reliable support forum's. In past, I have contributed to fedora-infra tahrir project and I must say that I had "great learning experience" while interacting with my mentor. I like fedora community and find everyone very helpful and motivating. In last few years, I have found some great projects has been started in fedora-infra like ptogit, shumgrepper, fresque etc, to which I would like to contribute.
Yes, I have contributed to fedora-infra tahrir project. I have made contributions to several other open source organizations i.e. Apache, Mozilla. Apart from these I have also made contribution to python projects on github i.e. Python-Cliff and Python-Click.
I have 3+ years experience in Python language which is basic required skill of the project. Fresque project is a flask app and I know the framework very well. Apart from these, I have experience in writing unit tests, handling database and also coming up new ideas. This project will involve development in flask, sqlalchemy, pygit2, unittests etc, which I know pretty well. I have already gone through the fresque project workflow and made few contribution to the project and thus, I think it makes me a strong candidate for this project.
I have made few contribution to the fresque project. Link to contribution is here
Yes, Last year I have participated for the Apache. The project was about developing command line application for Libcloud API. The project repository is available on on my github page.
I would love to work more with Fedora-Infra team because their projects intersect with my interests. I was working on Progit, Tahrir for some times and after GSOC, I would like to contribute to these projects as well as fresque.
In early May our summer vacation of college will start and ends by late of July; I can give my full time commitment to this project,. I assure dedication of at least 40 hours per week to the work and that I do not have any other obligations from early May till mid August. |
|
Past Experience |
|
In past I have made contribution to various projects of Mozilla and Apache foundation. Recently . Moving ahead, I have also written two open source libraries i.e.
which are available on pypi. Apart from this I have cofounded "Prequell" whose first project was scaling Flask application using blueprints and celery. Finally, I consider myself as an experienced Python developer as I have built various application across different domain using it.
|
|
Goal |
|
Fedora Fresque is a standard Python web application that abstracts away intricacies in package review process. Currently for any package to enter the fedora repository has to go through the review process which is mechanical and mannual, during this period contributors receives valuable feedbacks but once the package gets imported; all the information is lost. So, I will develop a web application which will expose dedicated RPM reviews using inline comments along with some level of automation. This will bring up lot of new possibilities to package manages and will allow them to connect to the packaging reality. |
|
Project Details |
|
The project will consist of 4 main phases: |
|
Phase 1: Add git server on backend |
|
The current process of package review have no git integration and with every new build of the package we looses the information of previous one. It means we are depriving the evolution of the spec file. Adding git will help us to track all the changes.
In this phase we will use python
def create_git_repo(name, gitfolder):
# Create a git project based on the package information.
gitrepo = os.path.join(gitfolder, '%s.git' % name)
if os.path.exists(gitrepo):
raise IOError('The project repo "%s" already exists' % name)
# create a bare git repository
pygit2.init_repository(gitrepo, bare=True)
return 'Project "%s" git respository created' % name
$ git clone git@fedorareviewserver.org:/var/git/doe/spiderman.git $ cd spiderman && touch spiderman.spec $ git add spiderman.spec $ git commit -m "initial commit" $ git push -u origin master
For every target release their will be separate branch in order to avoid the conflicts during review process. To keep it more general, the branch name will be given as fNN such as f14 for Fedora 14. This will be set to the default master branch.
Every new push will invoke fedora package review which will build the rpm for suitable target and the report generated will be sent via email as well as notification to the contributor. The fedora review server will have its own lookaside cache to speed up the build process or we can have dockers containers for it. |
|
Phase 2: Streamlining review process |
|
During package review (in which reviewers looks for the package before sending it to the fedora server) reviewer tries to identify bugs, gives feedback and keeps package more maintainable. For reveiw process, currently fedora packaging uses Bugzilla which is more like a bug tracker tool and feedback is given by creating Bugzilla ticket. Furthermore, most of the time a contributor is unclear for what he is waiting for. People forget about fedora flags and Bugzilla don't know about fedora relationship like who is contributor and who is the reviewer. So in this phase the task would be to add features which will bring features to remove the above drawbacks and to provides efficient and reliable review mechanism.
Watcher - who is interested in getting reports on package <br\> Contributor - who has created package that is being reviewed; <br\> Reviewer - One or more developers that review the package<br\> The only requirement is if it is the first package of a Contributor, the Reviewer must be in the Sponsor group and be willing to sponsor that Contributor.
Its time to look upon reviewer side,
Now we have seen that there is lack of streamline conversations i.e. reviewer can't comment where the issue is and each package update turns out to be more manual as contributor will have to follow aforesaid steps again. Fresque will introduce several new features to overcome the shortcomings
Example: Contributor Doe creates a spiderman python package and puts this on review process by providing informations about package, target builds, sponsor etc. The python packaging guidelines can found here. spiderman/
|--__init__.py
|--setup.py
|--db/
|--lib/
|--spider.spec
Currently, contributor uses their own spreadsheet and various other tools to keep track of changes. To remove this, all the comments and changes will be saved in the fedora-review database so that in future anyone can access it and this will remove the developer-developer dependencies. |
|
Phase 3: Fedora-Review integration |
|
In this phase I will integrate fedora-review because as of now, there is no automatic QE during review process. Fedora-Review implements API which removes many manual process in review process by doing automatic checks. Currently fedora review does following jobs
Integrating this will bring a great level of automation and will pave the way for continuos integration process for fedora packages. Implemention wise, fedora-review provides python API which can be easily amalgamated to the flask app on the backend system.  Lets look at one example,
# base image
from fedora:21
# install the dependecies and fedora-review
run yum install --assumeyes \
buildbot-slave git tar rpmdevtools \
gcc-c++ liblastfm-devel taglib-devel gettext boost-devel \
qt-devel cmake gstreamer1-devel gstreamer1-plugins-base-devel glew-devel \
libgpod-devel qjson-devel libplist-devel \
usbmuxd-devel libmtp-devel protobuf-devel protobuf-compiler qca2-devel \
libcdio-devel qca-ossl fftw-devel sparsehash-devel sqlite-devel \
pulseaudio-libs-devel libqtwebkit-dev sha2-devel desktop-file-utils \
libechonest-devel libchromaprint-devel python-pip fedora-review
run rpmdev-setuptree
run echo "fedora-21-64" > /fedora-review
# start the build process as mentioned in config file
cmd ["/usr/bin/python", "/config/slave/start.py"]
|
|
Phase 4: Unit tests and deployment |
|
Unit testing makes sure that our code works properly under a given set of conditions and assures correctness under a basic set of conditions. Syntax errors will almost certainly be caught by running tests, and the basic logic of a unit of code can be tested to ensure correctness under certain conditions. In this phase, I will write unit test for the flask application.
Example, unit tests for checking whether import json
import unittest
class FlaskTestCase(unittest.TestCase):
# using the unittest library, call the packages route from the app
# returned value contained in the JSON response
# match the result of those parameters
def test_packages(self):
tester = app.test_client(self)
response = tester.get('/packages', content_type='application/json')
self.assertEqual(response.status_code, 200)
# Check that the result sent is {'packages': ["spiderman"]}
self.assertEqual(json.loads(response.data), {"packages": ['spiderman']})
It will involve creating the proper stack on the server and also enabling proper security features like port forwarding and firewall. The following step will be followed for the deployment.
this can be shipped as docker container. The docker file to deploy flask app is available on my github page. |
|
Deriverables |
|
|
Timeline | |||||||||||||||||||||||||||||||
|
