Rahulrrixe (talk | contribs) No edit summary |
(Redirect Package Maintainer wiki links to docs.fp.o) |
||
| (11 intermediate revisions by 2 users not shown) | |||
| Line 13: | Line 13: | ||
* '''IRC nick''': rahulrrixe at irc.freenode.net | * '''IRC nick''': rahulrrixe at irc.freenode.net | ||
* '''Phone''': 91-9819737113 | * '''Phone''': 91-9819737113 | ||
* '''Fedora Profile''': [ | * '''Fedora Profile''': [[User:rahulrrixe|Rahul Ranjan]] | ||
* '''GitHub''': [https://github.com/rahulrrixe rahulrrixe] | * '''GitHub''': [https://github.com/rahulrrixe rahulrrixe] | ||
* '''Blog url''': https://medium.com/@RahulRRixe | * '''Blog url''': https://medium.com/@RahulRRixe | ||
| Line 28: | Line 28: | ||
|- | |- | ||
| style="border: 0px solid white;" | | | style="border: 0px solid white;" | | ||
* I’m a | * I’m a senior student at Indian Institute of Information Technology, Allahabad majoring in Information Technology. | ||
* I have excellent proficiency in Golang, Python and C/C++. I am passionate about building real time scalable web application. Furthermore, in past I have gained experienced in all the major web frameworks i.e. Flask, Django, ROR, and Martini. Apart from this, I am also comfortable with front-end technologies i.e. Javascript, CSS and HTML. | * I have excellent proficiency in Golang, Python and C/C++. I am passionate about building real time scalable web application. Furthermore, in past I have gained experienced in all the major web frameworks i.e. Flask, Django, ROR, and Martini. Apart from this, I am also comfortable with front-end technologies i.e. Javascript, CSS and HTML. | ||
* My Open Source journey started when I became Mozilla Student Ambassador and initially my contributions were limited to localization and community building. Later on it expanded to code contribution. Recently, I was contributing to release engineering groups projects. Links to some of my contributions are [https://github.com/mozilla/mozregression/commits?author=rahulrrixe mozregression] and [https://github.com/mozilla/build-relengapi/pull/173 releng API]. | * My Open Source journey started when I became Mozilla Student Ambassador and initially my contributions were limited to localization and community building. Later on it expanded to code contribution. Recently, I was contributing to release engineering groups projects. Links to some of my contributions are [https://github.com/mozilla/mozregression/commits?author=rahulrrixe mozregression] and [https://github.com/mozilla/build-relengapi/pull/173 releng API]. | ||
| Line 114: | Line 114: | ||
* Thereafter they can push the updated code to the fedora review server git repository. | * Thereafter they can push the updated code to the fedora review server git repository. | ||
* The changes data will be saved in the review server repository. | * The changes data will be saved in the review server repository. | ||
In this phase we will use python `pygit2` to interact with git. Also I will import most of the feature from [https://github.com/pypingou/pagure pagure] like keeping tracks of commits, revisions etc. | |||
For example: Doe submitted spiderman package for review and update his package with .spec file. The process using git will look like below. | For example: Doe submitted spiderman package for review and update his package with .spec file. The process using git will look like below. | ||
;on server side | |||
<pre> | <pre> | ||
$ git clone git@fedorareviewserver.org:/var/git/ | def create_git_repo(name, gitfolder): | ||
# Create a git project based on the package information. | |||
gitrepo = os.path.join(gitfolder, '%s.git' % name) | |||
if os.path.exists(gitrepo): | |||
raise IOError('The project repo "%s" already exists' % name) | |||
# create a bare git repository | |||
pygit2.init_repository(gitrepo, bare=True) | |||
return 'Project "%s" git respository created' % name | |||
</pre> | |||
;on client side | |||
<pre> | |||
$ git clone git@fedorareviewserver.org:/var/git/doe/spiderman.git | |||
$ cd spiderman && touch spiderman.spec | $ cd spiderman && touch spiderman.spec | ||
$ git add spiderman.spec | $ git add spiderman.spec | ||
| Line 149: | Line 166: | ||
* Then contributor do the required changes and again upload the package and if reviewers approves it, the flag get changed to '+'. | * Then contributor do the required changes and again upload the package and if reviewers approves it, the flag get changed to '+'. | ||
Now we have seen that there is lack of streamline conversations i.e. reviewer can't comment where the issue is and each package update turns out to be more manual as contributor will have to follow aforesaid steps again. Fresque will introduce several new features to overcome the shortcomings | Now we have seen that there is lack of streamline conversations i.e. reviewer can't comment where the issue is and each package update turns out to be more manual as contributor will have to follow aforesaid steps again. Fresque will introduce several new features to overcome the shortcomings | ||
* | * Threaded comments, inline discussions: This will allow to have discussion at individual line level in the file. | ||
;I have written a JavaScript code for selcting inline code text. Link to demo is [http://jsfiddle.net/rahulrrixe/1957n9mn/ here]. | |||
* Review Counts: This will show number of issues has been identified and requires modifications. | * Review Counts: This will show number of issues has been identified and requires modifications. | ||
* Review Tags: It will describe the type of review i.e. Bug, Enhancement etc. | * Review Tags: It will describe the type of review i.e. Bug, Enhancement etc. | ||
* Profiles of reviewers and contributors. | * Distributed teams: Profiles of reviewers and contributors. | ||
* Iterative reviews: Code review is inherently iterative. In this we keep tracking the newest revision and keeps it on top. | |||
Example: | Example: | ||
Contributor Doe creates a spiderman python package and puts this on review process by providing informations about package, target builds, sponsor etc. The python packaging guidelines can found [ | Contributor Doe creates a spiderman python package and puts this on review process by providing informations about package, target builds, sponsor etc. The python packaging guidelines can found [[Packaging:Python|here]]. | ||
<pre> | <pre> | ||
spiderman/ | spiderman/ | ||
| Line 182: | Line 199: | ||
|- | |- | ||
| style="border: 0px solid white;" | | | style="border: 0px solid white;" | | ||
In this phase | In this phase I will integrate fedora-review because as of now, there is no automatic QE during review process. Fedora-Review implements API which removes many manual process in review process by doing automatic checks. Currently fedora review does following jobs | ||
* Downloading SRPM & SPEC from Bugzilla report | * Downloading SRPM & SPEC from Bugzilla report | ||
* Build and install package in mock | * Build and install package in mock | ||
| Line 189: | Line 206: | ||
* Run rpmlint | * Run rpmlint | ||
* Generate review template with both manual & automated checks serving as a starting point for the review work. | * Generate review template with both manual & automated checks serving as a starting point for the review work. | ||
Integrating this will bring continuos integration process for fedora packages. | Integrating this will bring a great level of automation and will pave the way for continuos integration process for fedora packages. Implemention wise, fedora-review provides python API which can be easily amalgamated to the flask app on the backend system. | ||
[[File: | |||
[[File:Fresquenew.png|thumb|center|500px|alt=Architecture of the continuos integration of packages using fedora-review .|Flow diagram of the continuous integration system for the package using fedora-review and buildbot]] | |||
Lets look at one example, <br/> | Lets look at one example, <br/> | ||
After making changes when Doe push his code on the review server, these steps will follow up. | After making changes when Doe push his code on the review server, these steps will follow up. Our CI server is Taskotron and its buildbot master will be invoked for pakage build. | ||
* The taskotron server keeps running in background and will look for changes in the package. | |||
* if its detects changes in the package it will pass instruction to execute build process to its slaves. | |||
* These slaves as build bot will setup the appropriate environment and then it will notify to master as well as contributors. An example of Dockerfile of slave container. | |||
* The | |||
* if its detects changes in the | |||
* | |||
<pre> | <pre> | ||
# base image | # base image | ||
| Line 353: | Line 350: | ||
* [http://www.pygit2.org/ Pygit2 Documentation] | * [http://www.pygit2.org/ Pygit2 Documentation] | ||
* [http://blog.pingoured.fr/index.php?post/2011/11/12/Fedora-review Fedora-review, help yourself!] | * [http://blog.pingoured.fr/index.php?post/2011/11/12/Fedora-review Fedora-review, help yourself!] | ||
* [ | * [https://docs.fedoraproject.org/en-US/package-maintainers/Package_Review_Process/ Package Review Process] | ||
|} | |} | ||
[[category:Summer coding 2015]] | [[category:Summer coding 2015]] | ||
Latest revision as of 11:25, 9 October 2021
Project Title : Fresque
|
Contact Information |
|
|
About Me |
|
|
Q&A |
I have been using Fedora for past five years and I am fond of its user friendly interface and reliable support forum's. In past, I have contributed to fedora-infra tahrir project and I must say that I had "great learning experience" while interacting with my mentor. I like fedora community and find everyone very helpful and motivating. In last few years, I have found some great projects has been started in fedora-infra like ptogit, shumgrepper, fresque etc, to which I would like to contribute.
Yes, I have contributed to fedora-infra tahrir project. I have made contributions to several other open source organizations i.e. Apache, Mozilla. Apart from these I have also made contribution to python projects on github i.e. Python-Cliff and Python-Click.
I have 3+ years experience in Python language which is basic required skill of the project. Fresque project is a flask app and I know the framework very well. Apart from these, I have experience in writing unit tests, handling database and also coming up new ideas. This project will involve development in flask, sqlalchemy, pygit2, unittests etc, which I know pretty well. I have already gone through the fresque project workflow and made few contribution to the project and thus, I think it makes me a strong candidate for this project.
I have made few contribution to the fresque project. Link to contribution is here
Yes, Last year I have participated for the Apache. The project was about developing command line application for Libcloud API. The project repository is available on on my github page.
I would love to work more with Fedora-Infra team because their projects intersect with my interests. I was working on Progit, Tahrir for some times and after GSOC, I would like to contribute to these projects as well as fresque.
In early May our summer vacation of college will start and ends by late of July; I can give my full time commitment to this project,. I assure dedication of at least 40 hours per week to the work and that I do not have any other obligations from early May till mid August. |
|
Past Experience |
|
In past I have made contribution to various projects of Mozilla and Apache foundation. Recently . Moving ahead, I have also written two open source libraries i.e.
which are available on pypi. Apart from this I have cofounded "Prequell" whose first project was scaling Flask application using blueprints and celery. Finally, I consider myself as an experienced Python developer as I have built various application across different domain using it.
|
|
Goal |
|
Fedora Fresque is a standard Python web application that abstracts away intricacies in package review process. Currently for any package to enter the fedora repository has to go through the review process which is mechanical and mannual, during this period contributors receives valuable feedbacks but once the package gets imported; all the information is lost. So, I will develop a web application which will expose dedicated RPM reviews using inline comments along with some level of automation. This will bring up lot of new possibilities to package manages and will allow them to connect to the packaging reality. |
|
Project Details |
|
The project will consist of 4 main phases: |
|
Phase 1: Add git server on backend |
|
The current process of package review have no git integration and with every new build of the package we looses the information of previous one. It means we are depriving the evolution of the spec file. Adding git will help us to track all the changes.
In this phase we will use python
def create_git_repo(name, gitfolder):
# Create a git project based on the package information.
gitrepo = os.path.join(gitfolder, '%s.git' % name)
if os.path.exists(gitrepo):
raise IOError('The project repo "%s" already exists' % name)
# create a bare git repository
pygit2.init_repository(gitrepo, bare=True)
return 'Project "%s" git respository created' % name
$ git clone git@fedorareviewserver.org:/var/git/doe/spiderman.git $ cd spiderman && touch spiderman.spec $ git add spiderman.spec $ git commit -m "initial commit" $ git push -u origin master
For every target release their will be separate branch in order to avoid the conflicts during review process. To keep it more general, the branch name will be given as fNN such as f14 for Fedora 14. This will be set to the default master branch.
Every new push will invoke fedora package review which will build the rpm for suitable target and the report generated will be sent via email as well as notification to the contributor. The fedora review server will have its own lookaside cache to speed up the build process or we can have dockers containers for it. |
|
Phase 2: Streamlining review process |
|
During package review (in which reviewers looks for the package before sending it to the fedora server) reviewer tries to identify bugs, gives feedback and keeps package more maintainable. For reveiw process, currently fedora packaging uses Bugzilla which is more like a bug tracker tool and feedback is given by creating Bugzilla ticket. Furthermore, most of the time a contributor is unclear for what he is waiting for. People forget about fedora flags and Bugzilla don't know about fedora relationship like who is contributor and who is the reviewer. So in this phase the task would be to add features which will bring features to remove the above drawbacks and to provides efficient and reliable review mechanism.
Watcher - who is interested in getting reports on package <br\> Contributor - who has created package that is being reviewed; <br\> Reviewer - One or more developers that review the package<br\> The only requirement is if it is the first package of a Contributor, the Reviewer must be in the Sponsor group and be willing to sponsor that Contributor.
Its time to look upon reviewer side,
Now we have seen that there is lack of streamline conversations i.e. reviewer can't comment where the issue is and each package update turns out to be more manual as contributor will have to follow aforesaid steps again. Fresque will introduce several new features to overcome the shortcomings
Example: Contributor Doe creates a spiderman python package and puts this on review process by providing informations about package, target builds, sponsor etc. The python packaging guidelines can found here. spiderman/
|--__init__.py
|--setup.py
|--db/
|--lib/
|--spider.spec
Currently, contributor uses their own spreadsheet and various other tools to keep track of changes. To remove this, all the comments and changes will be saved in the fedora-review database so that in future anyone can access it and this will remove the developer-developer dependencies. |
|
Phase 3: Fedora-Review integration |
|
In this phase I will integrate fedora-review because as of now, there is no automatic QE during review process. Fedora-Review implements API which removes many manual process in review process by doing automatic checks. Currently fedora review does following jobs
Integrating this will bring a great level of automation and will pave the way for continuos integration process for fedora packages. Implemention wise, fedora-review provides python API which can be easily amalgamated to the flask app on the backend system.  Lets look at one example,
# base image
from fedora:21
# install the dependecies and fedora-review
run yum install --assumeyes \
buildbot-slave git tar rpmdevtools \
gcc-c++ liblastfm-devel taglib-devel gettext boost-devel \
qt-devel cmake gstreamer1-devel gstreamer1-plugins-base-devel glew-devel \
libgpod-devel qjson-devel libplist-devel \
usbmuxd-devel libmtp-devel protobuf-devel protobuf-compiler qca2-devel \
libcdio-devel qca-ossl fftw-devel sparsehash-devel sqlite-devel \
pulseaudio-libs-devel libqtwebkit-dev sha2-devel desktop-file-utils \
libechonest-devel libchromaprint-devel python-pip fedora-review
run rpmdev-setuptree
run echo "fedora-21-64" > /fedora-review
# start the build process as mentioned in config file
cmd ["/usr/bin/python", "/config/slave/start.py"]
|
|
Phase 4: Unit tests and deployment |
|
Unit testing makes sure that our code works properly under a given set of conditions and assures correctness under a basic set of conditions. Syntax errors will almost certainly be caught by running tests, and the basic logic of a unit of code can be tested to ensure correctness under certain conditions. In this phase, I will write unit test for the flask application.
Example, unit tests for checking whether import json
import unittest
class FlaskTestCase(unittest.TestCase):
# using the unittest library, call the packages route from the app
# returned value contained in the JSON response
# match the result of those parameters
def test_packages(self):
tester = app.test_client(self)
response = tester.get('/packages', content_type='application/json')
self.assertEqual(response.status_code, 200)
# Check that the result sent is {'packages': ["spiderman"]}
self.assertEqual(json.loads(response.data), {"packages": ['spiderman']})
It will involve creating the proper stack on the server and also enabling proper security features like port forwarding and firewall. The following step will be followed for the deployment.
this can be shipped as docker container. The docker file to deploy flask app is available on my github page. |
|
Deriverables |
|
|
Timeline | |||||||||||||||||||||||||||||||
|
