The websites arch (build, dev and trans)
Our build system
Our websites are rebuilt hourly. This is done using the syncStatic script which is puppet managed. In other words, you need to be on the infra team to change it. A sample copy is uploaded there. This is where we decide against which branch we build each websites. There is also a specific syncStatic.stg script for the staging websites. Example of stg websites are stg.fedoraproject.org and spins.stg.fedoraproject.org.
For release time, please refer to our specific documentation there. We have also a document which helps you not to forget anything when merging to the next release version.
Usage
To build the static websites pages you can run make from the specific website root directory. This will build the pages in all available languages. A list of all existing translations is available in the header of the Makefile.
This kind of building is rather long, if you want you can run also:
- make LANG: builds the pages in a specific language, where LANG is the two-letter abbreviation for the translation.
- make LANG test: builds a test instance at http://localhost:5000/ in a specific language, run make stoptest to take it down.
- make test: same as above, will build a test instance for all available languages.
Coding
Many of our websites are using the 960 Grid System which is why you will see many divs on our HTML sources. You might want to see an explanation.
Architecture
The fedora-web repository looks like the following (mostly all websites use the same tree as boot.fedoraproject.org).
.
|-- boot.fedoraproject.org
| |-- data
| |-- Makefile
| |-- po
| -- static
|-- build.d
| |-- buildconf.py
| |-- build.py
| |-- construct-translations.py
| |-- globalvar.py
| |-- pybabel.cfg
|
-- translations.ini
…
|-- httpd
|-- Makefile.in
`-- tools
Each websites directory consists on a data
folders holding HTML code, while all static code (JS, pictures, CSS…) are in static
. Some websites use specific building files in the build
dir if any.
build.d
contains the python building scripts and the Genshi template, as well as the language name localized for each luangages.
The httpd
folder contains standalone apache files to locally test our websites.
tools
is the place for manual script that we run times to time. In order to update the LINGUAS
file, or to check the websites..
i18n
We use the Genshi framework for internationalization. Our current templates need to have special markup around the strings in order to mark them as translatable.
There are several way to use the markups.
${_('String')}
${Markup(_('String with > html code'))}
${Markup(_('String with a <a href="%s">link</a>') % 'http://fedoraproject.org')}
${_('%(size)s, DVD ISO disc image for %(arch)s-bit PC') % {'size':'3.1 GB', 'arch':'32'}}
${Markup(_('If you think you\'ve found a bug, read <a href="%s">here</a> fist.') % ('http://fedoraproject.org/wiki/Common_F'+global_variables.release['prev_id']+'_bugs'))}
Fist you have the simplest example. Then the markup that could be used everywhere (it works with embedded html code, not as the first one). Then, you have an example using a variable. That is really useful in order to avoid having translators to translate again the string when changing the variable. The last example is a most complete one which shows you how to deal with multiple variables. The translator will be able to change the variable order using it's name.
To test that you haven't missed anything (for example you need to escape the simple quote from the string), try with make en test
in order to build the website for the en
language.
Once it's correct, you need to generate the new translation template file (POT) with make pot
.
For general information about Genshi templates, see:
- http://genshi.edgewall.org/wiki/Documentation, and in particular,
- http://genshi.edgewall.org/wiki/Documentation/xml-templates.html
Variables
As described in the examples above, you will note there are a lot of variables instead of hard codes. These variables have been set to make it easier to update our webpages. When merging from one branch to the next, i.e. from beta to final release, all the content of our pages can be updated by editing simply one single file, where all the variables are defined.
This file actually is used in the sites of fedoraproject.org and in spins.fedoraproject.org and is located in ~/build.d/globalvar.py
l10n
All POs are download twice a day. Any error should be reported to our tracking system. We download only the files translated at least at 10%. Please note that you won't be able to proofread only if you didn't get this far. This value is Makefile.in for the local test and in puppet/syncTranslation.sh for the online websites.
Whenever the HTML content change, we need to update the POT as explained on the previous section. Then, we need to push this POT to the translation platform, transifex.com. This is done by the command make pushpot
. But only the maintainers of the fedora-web release are able to push a new POT. Ask them if you need to do so. Please, open a new ticket.
In order to know if the builder is actually pulling the translations in a specific branch/website, check on the repo if a file named PO_FREEZE
exists on the website folder. This could be used while editing the Transifex POT without overriding the existing translations on production.
Adding a new language is done through 4 steps:
- first, create it at transifex.com by uploading a new translation on the website.
- Then, update the
translations.ini
file accordingly - update the
LINGUAS
file (tools/l10N_update.sh
might be of help) - update the local
httpd/conf/language.conf
file (by runningmake en test
for example) and ask to make this change in the infra (puppet). That is probably needed as we have the same exact copy of this file there. File a ticket for that on our Trac.
Joining/helping
Please read the Websites/Join and How to fix bugs pages.
Actual used branches for each (sub)domain
All our websites in production now are build against master branch. This makes it much easier to contributors to push their commits to the right branch. Staging websites on the other hand are build against the actual developing release-state branch: this could be alpha or beta, but in some cases can also be a specific branch. In fact we use the staging (stg) branch for specific enhancements, this allows us to keep the alpha or beta branch clean.
The new workflow means we will merge every single development branch to master at D-day, as production sites are build only against master branch.
This overview should help you to understand which git-branch is used to build a single subdomain or domain.
Domain | Branch |
fedoraproject.org | master |
stg.fedoraproject.org | f20-alpha |
spins.fedoraproject.org | master |
spins.stg.fedoraproject.org | f19 |
boot.fedoraproject.org | master |
fedoracommunity.org | master |
fadorahosted.org | master |
fedorapeople.org | master |
fudcon.fedoraproject.org | master |
start.fedoraproject.org | master |