L10N/Typo Purification of po Files

This page is a work in progress.

This page describes and explains how to setup and use a python tool that aims at detecting typographic faults in translation files. It assumed here that the typographic correction described here is part of a whole process of pulling, modifying and pushing translated files from/to Zanata. see [[]].

Description of the tool

User Interface

This tool is a python script that can be found at https://github.com/jaaf/po_purifier. It scan a directory for .po files. For each file, it checks translated messages against typographic rules that reside in a configuration file named typorules.py. Each time a typographic rule is not satisfied, the program stops and asks the user what to do. The figure 1 below shows how it looks like :

Figure 1: Typographic Fault Detected

• The message to the user, that appears in English here, normally appears in the user's language, provided that the program has been localized. It has 2 parts:
  • The first part that tells the user a typo rule is infringed and that he has to decide for change or not (this message is part of the program and has to be localized)
  • The typo rule itself (it belongs to the typorules.py file)

• In this case the French typo rule requires a narrow no break space between a value and its unit and the location of the fault is shown with a green highlight.

• To help the user, the message is shown twice. Firstly with the various spaces colorized according to their type, secondly with the typo fault highlighted in green color.

The figure 2 below shows what happens after the user has accepted the change.

Figure 2: Typographic Correction Accepted

• The message Change accepted is displayed.

• The corrected message is displayed in blue color.

• Then the program informs the user it has not changed some message because no typo faults were detected. It should do likewise till the next fault detection.

The figure 3 below shows a case where the user could use the c (for prior change) option. Indeed, we can see that an hyphen has been used in place of a semi-em dash. In French the spacing rules for hyphen and semi-em dash are different. An hyphen requires no space between the previous and the following word, while a semi-em dash requires a spaces for both. It appears that changing the hyphen with a semi-em dash is the best solution here.

Figure 3: The user should replace the hyphen with a semi-em dash

The following figures show how the process occurs.

Figure 4	Figure 5
Figure 6	Figure 7

Frequently, it happens that a message is not really a text in the target language but a very long output of command or the description of the command itself with a lot of options. In this case, the typo rules are very often infringed and the program stops many times. To avoid this the user can chose 's' to skip the message and prevent repetition of false detection. See fig 8 below.

Figure 8 - An example of message that should be skipped

Getting the tool

If you don't plan to improve the program, just go to The github page and use the green Clone or download button to download the po_purifier-master.zip file. Once unzipped, you should have a po_purifier-master folder. Inside this folder is a po_purifier folder that contains the program po_typo_purifier.py.

Aside this po_purifier folder is a fr folder. This folder contains a list of .po file that are here for testing purposes. The best is to take the po_purifier folder and to place here aside the folder that contains the translated files (.po) you have already pulled from Zanata.

Generally the folder that contains the .po file is named after your locale e.g. 'fr'. You should set the locale variable at the beginning of your typorules.py file with this value to allow the program to find the translated files.

If you chose to name the folder that contains the .po files otherwise, or to place it elsewhere, you have to use the -t (--trans-dir) option when calling the program to indicate the path of the .po files relative to where the parent folder of the program lays e.g.
python3 po_typo_purifier.py -t de/pot

Configuring the tool

The configuration of the tool for your language is done in the typorule.py file. You should set:

• the locale variable with your contry code as it is named by zanata e.g. 'fr', 'de', etc.
• the language code e.g. 'fr_FR.UTF-8' where the localized messages of the program are found under locale. Note that 'locale' is the name of a folder, not the locale variable above.
• the sr variable that contains the list of typographic rules for your language. Please take the French example as a guide for formatting these rules.

Using the tool

If the translation folder is aside the program's parent folder po_purifier and contains the .po file at its first level, just use:

python3 <path_to_program>/po_typo_purifier.py 

Otherwise use:

python3 <path_to_program>/po_typo_purifier.py -t <path_to_po_file> 

where<path_to_po_file> is the path to the .po file starting from the program's parent folder po_purifier e.g.

python3 <path_to_program>/po_typo_purifier.py -t de/pot

Finding the results

Normally the original files remain unchanged and the corrected files (or not corrected for some of them) are placed in a folder named purified aside the program's parent folder po_purifier.

Don't forget to delete this folder if you run the program anew.

Of course, if you agree with the changes, you should replace the original files in your pull, modify, push from/to zanata process with these modified files and pursue your modifications manually or with others tools.