From Fedora Project Wiki
No edit summary
No edit summary
Line 125: Line 125:
For every target release their will be separate branch in order to avoid the conflicts during revive process. To keep it more general, the branch name will be given as fNN such as f14 for Fedora 14. This will be set to the default master branch.
For every target release their will be separate branch in order to avoid the conflicts during revive process. To keep it more general, the branch name will be given as fNN such as f14 for Fedora 14. This will be set to the default master branch.
;Reporting
;Reporting
Every new push can invoke fedora package review which will build the rpm for suitable target and the report generated will be sent via email to the packager.  
Every new push can invoke fedora package review which will build the rpm for suitable target and the report generated will be sent via email to the packager. The review server will have its own lookaside cache to speed up the build process.


|- style="background-color: #e6e6e6;"
|- style="background-color: #e6e6e6;"
Line 132: Line 132:
|-  
|-  
| style="border: 0px solid white;" |
| style="border: 0px solid white;" |
During package review (in which reviewers looks for the package before sending it to the fedora server) reviewer tries to identify bugs, gives feedback and keeps code more maintainable. So to facilitate this currently fedora packaging review uses Bugzilla which is more like a bug tracker tool than reviewer and feedback is given by creating Bugzilla ticket. Furthermore, we don't have support inline comments on the packages files. So in this phase the task would be to add features which will facilitates efficient and reliable bidirectional review channel.
During package review (in which reviewers looks for the package before sending it to the fedora server) reviewer tries to identify bugs, gives feedback and keeps package more maintainable. So to facilitate this currently fedora packaging review uses Bugzilla which is more like a bug tracker tool than reviewer and feedback is given by creating Bugzilla ticket. Furthermore, contributor is unclear for what he is waiting for. People forget about fedora flags and Bugzilla don't know about fedora relationship like who is contributor and who is the reviewer. So in this phase the task would be to add features which will bring features to remove the above drawbacks and will facilitates efficient and reliable review mechanism.  
;The review process will be distributed as
 
;The review process constitutes  three entities:
<strong>Watcher</strong> - who is interested in getting reports on repository <br\>
<strong>Watcher</strong> - who is interested in getting reports on repository <br\>
<strong>Contributor</strong> - who has created module/code that is being reviewed; <br\>
<strong>Contributor</strong> - who has created module/code that is being reviewed; <br\>
Line 172: Line 173:
* After that John goes ahead and checks that the modifications were done and he changes the review flag to `+`.
* After that John goes ahead and checks that the modifications were done and he changes the review flag to `+`.


All the comments and changes are saved in the fedora-review database so that in future anyone can access it. Here we will remove the developer dependencies.
Currently, contributor uses their own spreadsheet and various other tools to keep track of changes to removes this, all the comments and changes are saved in the fedora-review database so that in future anyone can access it. Here we will remove the developer-developer dependencies.


|- style="background-color: #e6e6e6;"
|- style="background-color: #e6e6e6;"
Line 179: Line 180:
|-  
|-  
| style="border: 0px solid white;" |
| style="border: 0px solid white;" |
No automatic QE during review. Fedora-Review implements API which removes many manual process in review process by doing automatic checks.
In this phase we will integrate fedora-review because as of now, there is no automatic QE during review process. Fedora-Review implements API which removes many manual process in review process by doing automatic checks. Currently fedora review does following jobs
 
* Downloading SRPM & SPEC from Bugzilla report
People use own spreadsheet and other tools to keep track of changes.
* Build and install package in mock
 
* Download upstream source
unclear for what you are waiting for. People forget about fedora flags. Bugzilla don't know about fedora relationship. like who is packager and who is the reviewer.
* Check md5sums
 
* Run rpmlint
Review Server: Git enabled server which also contains lookaside cache and tools and automation tools.
* Generate review template with both manual & automated checks serving
every push will upload new tarball
  as a starting point for the review work.
 
Integrating this will bring continuos integration process for fedora packages. To implement this, fedora review provides python API and that can be easily amalgamated to the flask app on the backend system.


|- style="background-color: #e6e6e6;"
|- style="background-color: #e6e6e6;"
| style="border: 0px solid white;" |
| style="border: 0px solid white;" |
'''Phase 4: user friendly GUI and robust unit tests'''
'''Phase 4: Unit tests and deployment'''
|-  
|-  
| style="border: 0px solid white;" |
| style="border: 0px solid white;" |
 
Unit testing makes sure that our code works properly under a given set of conditions and assures correctness under a basic set of conditions. Syntax errors will almost certainly be caught by running tests, and the basic logic of a unit of code can be tested to ensure correctness under certain conditions
|}
|}



Revision as of 15:58, 25 March 2015

Project Title : Fresque

Contact Information

About Me

  • I’m a Senior student at Indian Institute of Information Technology, Allahabad majoring in Information Technology.
  • I have excellent proficiency in Golang, Python and C/C++. I am passionate about building real time scalable web application. Furthermore, in past I have gained experienced in all the major web frameworks i.e. Flask, Django, ROR, and Martini. Apart from this, I am also comfortable with front-end technologies i.e. Javascript, CSS and HTML.
  • My Open Source journey started when I became Mozilla Student Ambassador and initially my contributions were limited to localization and community building. Later on it expanded to code contribution. Recently, I was contributing to release engineering groups projects. Links to some of my contributions are mozregression and releng API.

Q&A

Why do you want to work with the Fedora Project?

I have been using Fedora for past five years and I am fond of its user friendly interface and reliable support forum's. In past, I have contributed to fedora-infra tahrir project and I must say that I had "great learning experience" while interacting with my mentor. I like fedora community and find everyone very helpful and motivating. In last few years, I have found some great projects has been started in fedora-infra like ptogit, shumgrepper, fresque etc, to which I would like to contribute.

Do you have any past involvement with the Fedora project or another open source project as a contributor?

Yes, I have contributed to fedora-infra tahrir project. I have made contributions to several other open source organizations i.e. Apache, Mozilla. Apart from these I have also made contribution to python projects on github i.e. Python-Cliff and Python-Click.

Why should we choose you over other applicants?

I have 3+ years experience in Python language which is basic required skill of the project. Fresque project is a flask app and I know the framework very well. Apart from these, I have experience in writing unit tests, handling database and also coming up new ideas. This project will involve development in flask, sqlalchemy, pygit2, unittests etc, which I know pretty well. I have already gone through the fresque project workflow and made few contribution to the project and thus, I think it makes me a strong candidate for this project.

Contribution to Fresque Project

I have made few contribution to the fresque project. Link to contribution is here

Did you participate with the past GSoC programs, if so which years, which organizations?

Yes, Last year I have participated for the Apache. The project was about developing command line application for Libcloud API. The project repository is available on on my github page.

Will you continue contributing/ supporting the Fedora project after the GSoC 2015 program, if yes, which team(s), you are interested with?

I would love to work more with Fedora-Infra team because their projects intersect with my interests. I was working on Progit, Tahrir for some times and after GSOC, I would like to contribute to these projects as well as fresque.

Will you have any other time commitments, such as school work, another job, planned vacation, etc., during the duration of the program?

In early May our summer vacation of college will start and ends by late of July; I can give my full time commitment to this project,. I assure dedication of at least 40 hours per week to the work and that I do not have any other obligations from early May till mid August.

Past Experience

In past I have made contribution to various projects of Mozilla and Apache foundation. Recently . Moving ahead, I have also written two open source libraries i.e.

  • Parinx: Sphinx docstring parser language which provides a interface to extract the relavant parameter.
  • Sec-Edgar:Download all companies periodic reports, filings and forms from EDGAR database,

which are available on pypi. Apart from this I have cofounded "Prequell" whose first project was scaling Flask application using blueprints and celery. Finally, I consider myself as an experienced Python developer as I have built various application across different domain using it.

  • Prior to GSOC, I have completed internship at The Walt Disney Company and their I have worked on building highly personalized real-time news application using Golang and Python.

Goal

Fedora Fresque is a standard Python web application that abstracts away intricacies in package review process. Currently for any package to enter the fedora repository has to go through the manual review process, during this period they receive valuable feedbacks but once the package gets imported; all the information is lost.

So, I will develop a web application which will expose dedicated RPM reviews using inline comments along with some level of automation. This will bring up lot of new possibilities to package manages and will allow them to connect to the packaging reality.

Project Details

The project will consist of 4 main phases:

Phase 1: Add git server on backend

The current process of package review have no git integration and with every new build of the package we looses the information of previous one. It means we are depriving the evolution of the spec file. Adding git will help us to track all the changes.

The process workflow can be described as
  • When ever a new request for package review will be requested, a bare git repository with same name as of the package gets created on the fedora review server.
  • The bare repository keeps tracks of the changes made by the packagers.
  • Now packager can clone this git repository on their local machines and can make changes to it.
  • Thereafter they can push the updated code to the fedora review server git repository.
  • The changes data will be saved in the review server repository.

For example: Rahul submitted spiderman package for review and update his package with .spec file. The process using git will look like below.

$ git clone git@fedorareviewserver.org:/var/git/rahul/spiderman.git
$ cd spiderman && touch spiderman.spec
$ git add spiderman.spec
$ git commit -m "initial commit"
$ git push -u origin master
Branching

For every target release their will be separate branch in order to avoid the conflicts during revive process. To keep it more general, the branch name will be given as fNN such as f14 for Fedora 14. This will be set to the default master branch.

Reporting

Every new push can invoke fedora package review which will build the rpm for suitable target and the report generated will be sent via email to the packager. The review server will have its own lookaside cache to speed up the build process.

Phase 2: Streamlining review process

During package review (in which reviewers looks for the package before sending it to the fedora server) reviewer tries to identify bugs, gives feedback and keeps package more maintainable. So to facilitate this currently fedora packaging review uses Bugzilla which is more like a bug tracker tool than reviewer and feedback is given by creating Bugzilla ticket. Furthermore, contributor is unclear for what he is waiting for. People forget about fedora flags and Bugzilla don't know about fedora relationship like who is contributor and who is the reviewer. So in this phase the task would be to add features which will bring features to remove the above drawbacks and will facilitates efficient and reliable review mechanism.

The review process constitutes three entities

Watcher - who is interested in getting reports on repository <br\> Contributor - who has created module/code that is being reviewed; <br\> Reviewer - One or more developers that review the module. <br\>

The minimum number of reviewer would be one and watcher can be more than one.

The packages review workflow process can be summarized as following:

  • Contributor creates the package.
  • Then he creates a spec file for the package, where he states the package and build that is being reviewed.
  • The reviewer set the flag to '?', which means it is under review and he identifies the bugs and issues.
  • After the contributor has made the required changes, he again upload the new package and reviewers approves it, the flag get changed to '+'.

So as we can see, there is no streamlining of conversation which means the reviewer can't comment where the issue is. Uploading different tar-ball and spec file with each review turnout to be very mechanical and manual. So now to improve this, fedora-review server will introduce several features:

  • Inline Comments: These comments will allow to have discussion on line specific modifications in the files
  • Q&A: This will be for broad discussions on particular file.
  • Review Counts: This will show number of issues has been identified and requires changes or modifications.
  • Review Tags: It will describe the type of review i.e. Bug, Enhancement etc.

Example:

Contributor Rahul creates a spiderman package and puts this on review process.

spiderman/
	|--__init__.py
	|--setup.py
	|--db/
	|--lib/
	|--spider.spec
  • He puts his package for review. The reviewer John will set the flag to ?.
  • John comments on the current setup.py for adding dependencies.
  • Reviewer may schedule a meeting here or can comment and after meeting takes place and the defects are marked on the review server. The total count of reviews is also included.
  • Rahul goes and makes the changes on the source code and push the modified revision to the fedora review server.
  • Now Rahul generates new spec and marks off the bug.
  • After that John goes ahead and checks that the modifications were done and he changes the review flag to +.

Currently, contributor uses their own spreadsheet and various other tools to keep track of changes to removes this, all the comments and changes are saved in the fedora-review database so that in future anyone can access it. Here we will remove the developer-developer dependencies.

Phase 3: Fedora-Review integration

In this phase we will integrate fedora-review because as of now, there is no automatic QE during review process. Fedora-Review implements API which removes many manual process in review process by doing automatic checks. Currently fedora review does following jobs

  • Downloading SRPM & SPEC from Bugzilla report
  • Build and install package in mock
  • Download upstream source
  • Check md5sums
  • Run rpmlint
  • Generate review template with both manual & automated checks serving
 as a starting point for the review work.

Integrating this will bring continuos integration process for fedora packages. To implement this, fedora review provides python API and that can be easily amalgamated to the flask app on the backend system.

Phase 4: Unit tests and deployment

Unit testing makes sure that our code works properly under a given set of conditions and assures correctness under a basic set of conditions. Syntax errors will almost certainly be caught by running tests, and the basic logic of a unit of code can be tested to ensure correctness under certain conditions

Deriverables

  • Web front-end and backend of fresque
  • Deployment of the app
  • Documentation and Manual
  • Unit Tests
  • Integrating of the app with other applications

Timeline

Period Task
April 27 - May 14 Community bonding, reading documentation and getting familiar with all the codes.
May 15 - May 24 Writing unit tests for the previous already written functions and fixing bugs.
May 25 Official GSoC coding period begins.
May 25 - June 09 (2 weeks) Git Backend Phase - Development of fedora review git server.
June 10 - June 17 (1 week) Add HTML file browser for git repository.
June 18 - June 25 (1 week) Review Process Phase - Adding backend functions for review process.
June 25 - July 03 Mid term evaluation period.
July 04 - July 14 (10 days) Front-end interface development for the review process.
July 15 - July 31 (2 weeks) Fedora-Review integration Phase: add fedora-review tool for automatic testing of new reviews
Aug 01 - Aug 05 (5 days) Web-Front end development which involve creating user friendly design and embedding security features.
Aug 05 - Aug 12 (1 week) Writing unit tests
August 13 - August 20 (1 week) Sanitizing codes, documenting everything, reviewing all the functionalities

and fixing bugs.

August 21- August 27 (1 week) Pencils down period. Submitting the project for final evaluation.

References