Translations Infrastructure SOP

This is still not official/standarized, at least until this notice is removed.

Translators contribute to a multitude of ever-changing projects and repositories, and need a central place/tool to give them a high-level view of their work. We provide Transifex and Damned Lies on http://translate.fedoraproject.org/ to serve that purpose.

Contact Information

Owner:: Fedora Infrastructure Team, Fedora Translation Team

Contact:: DimitrisGlezos, MikeMcGrath, RickyZhou, #fedora-admin, sysadmin-web group

Servers:: app1, app2 (stats), app3, app4 (transifex)

Purpose:: Provides translators front-ends to translation statistics (damned-lies) and submissions (Transifex) to various VCSs

Description

Translators contribute to a multitude of ever-changing projects and repositories, and need a central place/tool to give them a high-level view of their work. To make things worse, in Fedora we support >1 types of Version Control Systems (VCSs). The tools we provide to make the work of the Fedora Localization Project (FLP) easier include:

1. An instance of GNOME's Damned Lies, deployed at http://translate.fedoraproject.org/, which provides translation statistics (pull) for all the modules the FLP is interested in. Our templates differ significantly from upstream, but we try not to alter python files downstream (we prefer to submit patches upstream), although some python files do have some very small changes.
2. An instance of Transifex (source , deployed at https://translate.fedoraproject.org/submit/, which provides a submission interface (push) for translators (members of the 'cvsl10n' group).

Damned Lies

• Relevant contacts: DimitrisGlezos, DiegoZacarao.
• Important files are some xml.in files in /cvs/l10n/web/flpweb/.
• See also: L10N/ResourceMaintainers .

Add a module to DL

1. Check anonymous checkout

   cd temp
   git clone http://...
   

2. Check out (or update) flpweb:

   export CVSROOT=:ext:FEDORA_USERNAME@cvs.fedoraproject.org:/cvs/l10n
   cvs co web
   cd web/flpweb
   <!--# OR just update:
   -->
   cd web/flpweb
   cvs -n up
   cvs up -C
   

3. Add relevant entries to people.xml.in , fedora-modules.xml.in , releases.xml.in ).
4. Test changes:

   make
   python update-stats.py <modulename>
   

   

   Open up browser at http://flpweb.localhost/module/MODULENAME and make sure stats show up OK.
   
5. Update translations

   cd po
   ls *po | cut -f1 -d'.' | xargs -I'{}' intltool-update -d '{}'
   cd ..
   

6. Commit, mark as LIVE:

   cvs commit -m 'Added module foo. Tested, marking as LIVE'
   cvs tag -F LIVE
   

7. Let people know: Send an email to fedora-trans-list and let translators know, or ask the maintainer to do it (better).

Transifex: Setup submissions (eponymous checkout/in)

• Relevant contacts: DimitrisGlezos, MikeMcGrath, RickyZhou.

Part 1: Setup the keys

If the repository is on Fedora Hosted Projects, then have a group administrator add the "transif" user to their FAS group.

1. SSH to the puppet server

   ssh bastion.fedoraproject.org
   ssh puppet1
   

2. Create an SSH key and add it to the private CVS repository:

   N='MODULENAME'; ssh-keygen -t dsa -C transifex-$N -f id_dsa-$N
   

3. Add the SSH key filename to configs/web/applications/transifex-ssh-add.sh.
4. Send the public key of the key to the module owner and ask him to create a user on his VCS with this public key. If they can, they may restrict the user's access only to certain files (eg. in the po/ subdirectory). Continue to the next step when you receive an affirmative reply.

Part 2: Check access

1. SSH again as last step.
2. Check eponymous checkout:

   ssh app3
   sudo -H -u transifex -s
   export SSH_AUTH_SOCK=/var/lib/transifex/ssh-agent-sock-transifex
   ssh-add ~/.ssh/id_dsa-REPONAME
   cd temp
   git clone git+ssh://...
   git checkout BRANCHNAME
   

3. Check eponoymous checkin:

   cd MODULENAME/po
   git commit -m 'Testing Transifex instance on Fedora with a very minor change. Please ignore.'
   git push
   

Part 3: Add to transifex

1. Navigate to https://translate.fedoraproject.org/submit/repo/add and add the repo.
2. Navigate to https://translate.fedoraproject.org/submit/module/add and add the module as disabled. ( Sometimes the repo isn't added and the server needs restart!?)
   • If the branch name doesn't exist in the list:

   tg-admin shell
   m=Module.selectBy(name='yumex')[0] 
   m.name
   m.branches
   b=Branch(name='yumex-2.0')
   m.addBranch()
   

3. Open up the console and check that the module checks out correctly:

   tg-admin shell
   m=Module.selectBy(name='yumex')[0] 
   m.name
   m.checkout()
   m.disabled = False
   

4. Login to the web front-end and commit a file. https://translate.fedoraproject.org/submit/module/MODULENAME

Part 4: Let people know

1. Send an email to the module maintainer and let him know
2. Send an email to fedora-trans-list and let translators know, or ask the maintainer to do it (better).

Troubleshooting and Resolution

Module statistics in DL show various errors and warnings

Most likely the module provides a POT file instead of having i18n support through intltool, which can build the POT on the fly, list available languages in po/LINGUAS etc. This is usually normal, although it could be considered a good opportunity to remind developers to use intltool. :)

Module statistics in DL are inexistent or weird

Someone added a module without checking its correctness with python update-stats.py <modulename>. XML files need editing.

Start/stop transifex service

You will need the transifex SSH key passphrase for this.

# If ssh-agent isn't already running:
sudo -u transifex /srv/tg/transifex/ssh-add.sh

ssh app{3,4}

sudo supervisorctl start transifex

sudo supervisorctl stop transifex

sudo supervisorctl restart transifex

Update only a few modules

#!/bin/bash

export MODULES='module1 module2'
export BACKUPDIR=/var/tmp/l10n-data/scratchdir/temp/

mkdir -p $BACKUPDIR
cd /srv/web/translation/

for m in $MODULES
do
mv /var/tmp/l10n-data/scratchdir/{cvs,svn,hg,git}/$m.* $BACKUPDIR
python update-stats.py $m /var/log/fedora-l10n/update-stats.<code>date +\%F</code>.log
done