SummerOfCode/2007/VillePekkaVainio

Man/info page wiki editor

• Student: Ville-Pekka Vainio
• Mentor: Karsten Wade

Introduction

This is not a Google Summer of Code project, this is under a similar Finnish event called COSS Summercode 2007. Here are some links about the event:

• COSS Main Page
• Summercode Finland 2007
• Summercode Finland FAQ
• My project page at MoinMoin wiki
• My project blog

During the summer I will be extending MoinMoin with man and info page publication and editing capabilities. The idea came from FedoraBounties, "Publication of all man and info pages for each release". There is also a GSoC 2007 project based on the same idea, but with different implementation plans, see

Planned features

• Import man/info pages from different releases, FC6, F7, F8, etc.
• Also man/info pages from released updates and Rawhide should be included
• Clean URLs that can be used as a reference
• Searching from those pages (this comes pretty much automatically from MoinMoin, may require some tweaking)
• Comparing and taking diffs from pages between releases, released updates and Rawhide (basics come from MoinMoin but requires work)

• Regular users can edit the man/info pages in the wiki
• Administrators can accept or deny users' edits
• Admins can take a specific set of edits that they want to send upstream
• The wiki helps them to get a diff of those edits and make a Bugzilla report upstream

Preliminary schedule

Some basic points here, I'll add more details as the project goes on. I have reserved 13 weeks for my project, calendar weeks 23 — 35

Week 23	First week, research MoinMoin, inform the community about my project, get a development environment working, revise on Python & PEP8, first test programs, look at how to handle CVS & RPM from Python
Week 24	Start implementing the fetching of man/info pages from CVS to Moin, Moin version & internal storage format should be decided
Weeks 25 — 27	Implementation, change Moin's diff tool so that the "same" man/info pages from different distro versions can be compared
Milestone 1	Importing man/info pages should be done by July 6. Could slip to July 13 if absolutely necessary, preferably not.
Weeks 28 (29) — 33	Implement the editing of man/info pages, first possibility is to have the wiki export basic txt diffs for Bugzilla reports, the better (and more difficult) option is to have actual diffs against the man/info sources.
Milestone 2	All features should be done by August 17. If not, then at least apply feature freeze for the final 2 weeks.
Week 34	Mainly testing and bug fixing, if the schedule is delayed, then some more coding
Week 35	Final week, testing, reporting etc.

Open questions and talking points

Please see discussion on these points in the Moin wiki page of this project . I may not keep these two pages "synchronized". Instead, I suggest that MM specific points are discussed on their wiki and Fedora specific points here.

Internal storage format of man/info pages?

• Basic txt format and then generate textual diffs for Bugzilla reports
• Probably the easiest to implement but requires manual work from upstream people
• *roff, DocBook, TeX, etc.
• These are more difficult to implement but it could be maybe possible to then generate diffs that upstream could merge more easily

This is somewhat of a problem now. If we take only man pages into consideration here to begin with, the publication of those is pretty easy. I could write a Moin parser for man source. The editing is lot more difficult

• If the man pages are stored into Moin "as-is", then generating patches would be easy, but editing the pages would require *roff knowledge, not easy for the regular users
• The pages could be imported and stored as wiki markup, which would be probably the easiest format for regular users to edit. But producing meaningful, clean patches against upstream man source would be quite difficult.
• Even if I made a wiki markup -> man source exporter, the diffs might not be usable, since the computer generated man source could be too different from the original upstream source.
• The main point here is this: What do we want from this system? Do we want it to be a source of information and ideas for the man page maintainers (maybe by sending notification emails from wiki edits) or a tool that produces patches which can be applied with minimal human interaction?
  

After some discussion on #fedora-docs, the idea is to store the man/info pages as wiki markup and then convert those to DocBook XML when exporting. That XML can then be transformed into whichever different format is needed (see eg. http://docbook2x.sourceforge.net/)

• During the first phase (publication) doclifter will be used to store the man/info pages as DocBook XML. Moin can parse that XML so that it's viewable as "normal" wiki pages.

Where to look for these man/info files?

• Repositories, CVS, etc.
• The system should handle a lot of different sources and be extendable for different version management systems and package managers
• Remember, this is something that we would like other distros to use eventually, too :)

Diff tool enhancements

• Moin's diff tool probably has to be changed somehow to support diffs between different pages, not just different revisions of the same page.
• Or do we need / should we have a better versioning scheme in general, where Moin could have multiple "main" versions of a page?

• Probably more points to come during the summer ;)

TODO

Here's a list of things to be done soon, this may not be in any particular order, just a reminder to me and the people watching this project.

• As ThomasWaldmann suggested on #moin-dev, contact man/info maintainers and ask what they would want and need from this kind of system.
• Linux man-pages project (I've contacted Michael Kerrisk already)
• GNU Texinfo project (I should maybe get to know texinfo a bit better before discussing it on their mailing list)
• Maybe fedora-devel to hopefully reach developers who maintain Fedora packages and man pages in those packages (My mail to fedora-devel-list )

Phase 1

1. Test doclifter 1. Make an importer for Moin that converts man source into DocBook XML through doclifter and saves the results in a clean namespace hierarchy 1. Modify Moin's diff functionality or make an action that takes the diff of two different pages 1. Extend the importer to handle info pages too. The conversion can be made through GNU makeinfo, but it needs Texinfo sources, it doesn't work on Info sources.

Completed

• Which MoinMoin version to base the work on?
• 1.6 is the next stable, that will probably be taken into production use at Fedora a while after its release
• It's possible that these changes won't make it to 1.6 upstream anymore, since that is aimed to be stable soon
• 1.7 is the development version that all MoinMoin GSoC students use
• It could be easier getting these changes eventually merged into 1.7 upstream
• It'll take time until 1.7 is ready and released, I think the timeline is "this year"
• It'll maybe take even longer before Fedora upgrades to 1.7

• Finally decide on 1.6 vs. 1.7. This needs to happen this week as of writing (week 23). No major changes will get to 1.6 anymore, so if I based my project on it, then Fedora would need to run an unmerged branch of the code. Probably not what we want? My suggestion is to work on 1.7 branch and hopefully get my changes merged into mainline eventually.

• (./) 1.7 will be used, see the corresponding fedora-infrastructure-list thread .

• It's decided that I'll use 1.7, so now talk to the Moin developers about maybe getting a Mercurial branch for my project on http://hg.thinkmo.de/
• (./) My project has a repository now, at http://hg.thinkmo.de/moin/1.7-maninfo-vpv/

• Introduce this project and myself on fedora-docs-list, fedora-websites-list and maybe fedora-infstructure-list
• (./) fedora-docs-list self-introduction , project introduction on fedora-websites-list

Code

The code will be kept in MoinMoin's Mercurial, in a separate 1.7 branch. The address of the repository is [1] , see Moin's Mercurial guide on how to get and update source code from Mercurial.

Fedora has packaged Mercurial, so it can be installed through yum/pirut etc. easily.

About me

You can find some info about me from my Wiki page , too.

I'm a Computer Science student from University of Helsinki . I have taken all the courses required for a B.Sc. degree, but haven't officially graduated yet. My B.Sc. thesis was about Generics in Java, C++ and C#. I'll continue my M.Sc. studies after the summer.

I've been programming for over ten years and using Fedora since FC2. I'm one of the Finnish Fedora translators.

Comments, discussion

If you have any comments and ideas, please feel free to add them here :) This project can also be discussed on docs-list or #fedora-docs. I'm reading both of them.