View source for Editing audacityteam.org

{{Intro|This page describes the process in place to make changes to the main Audacity website on http://audacity.sourceforge.net/. This process requires that you have a SourceForge account and are listed as a developer on the Audacity project.|}}


== How the site works ==
In order to make the site available in many languages, we use PHP and the GNU gettext tools to deliver one set of pages translated into many languages. This makes it possible to update all language versions of the web site with minimal effort. Text that has not yet been translated into a given language will appear in English until the translation is updated.

The site has a single set of PHP pages which contain the English version of the pages. All text that appears on the screen is stored as PHP strings (rather than using included HTML) and output using ''='', ''echo'' and similar. These strings are marked for translation by enclosing them in brackets preceded by an underscore.
 _('some text');

When the pages are altered, '''xgettext''' is used to go through all the .php pages on the site and construct a message catalogue of all the strings marked as needing translation. Duplicates and known standard strings (dates and so on) are removed at this point to create audacity_website.pot, which is the translation template. A copy of this template exists for every language the web site is translated into, named after the ISO language code with a .po extension (so fr.po, de.po).... These contain all the strings marked for translation and the translation into that language of the string. These files are maintained and updated by the team of Audacity [[Contribute#Translators|translators]] for each language.

Once translations have been obtained, they are compiled using '''msgfmt''' to create message catalogue files (*.mo files) for each language. Localized pages are served automatically based on the Accept-Language header sent by the user's browser, or can be selected manually using language links in the page footer (this method also sets a cookie that serves future pages in that language). When the page is is sent to the visitor, the web server reads the relevant .mo file, looks up each string, and sends the translated text to the user instead of the English version.


== Editing the website ==
Audacity keeps the web site's PHP source in [http://code.google.com/p/audacity/source/browse/website/trunk/htdocs/ SVN]. So to edit the web site you need to check out a working copy of the source from the ''website'' directory in SVN. You can then edit the relevant page, and commit your changes back to SVN. At this point, nothing has changed on the web server, but the new version of the site has been stored in SVN.
{{hint|For complex changes, you may want to upload or create test pages before committing and pushing them. Don't forget to remove test pages when finished.  
* On Windows, [http://winscp.net/eng/index.php WinSCP] can be used under SCP or SFTP protocol or [http://filezilla-project.org Filezilla] under SFTP. See http://sourceforge.net/apps/trac/sourceforge/wiki/Project%20web .
* On a *nix system, you can ''make pull'' (remember if you have local modifications this will wipe them out) and ''make push'' to move any files on and off the server, even if they are not in SVN.}}<br>

== Updating the web server from SVN ==
The copy of the web site that users actually see is located on SourceForge servers. In order for changes in SVN to become visible, they need to be transmitted there. '''This process currently only works on *nix-like systems (such as Linux, or Mac OS X if you use the command line tools). Users without a *nix workstation can perform the same steps on the [https://sourceforge.net/p/forge/documentation/Shell%20Service/ Sourceforge Shell Service] by logging in over SSH.'''
=== Prerequisites ===
To update the web site you will need all of the following installed on your system. Note that at this point you are not actually editing anything, so a remote shell login to a system (e.g. the [https://sourceforge.net/p/forge/documentation/Shell%20Service/ Sourceforge Shell Service], which has all the required tools) is quite sufficient.
* '''SVN  (aka Subversion)'''
* '''SSH:''' Needed as a transport layer for RSync.
* '''RSync''' Used to transmit files to and from the SourceForge web server.
* '''GNU Make:''' Used to automate commands in the translation process. Other makes may work but have not been tested.
* '''GNU gettext tools:''' '''msgfmt''' and '''xgettext''' are needed to generate and compile translations.
* '''Python:''' Audacity uses a customised message merging tool to deal with cases where a string is not translated but a very similar one is. This is written in Python, so your system needs to be able to run ''.py'' scripts.

You also need a working copy of the ''website'' SVN directory. If you are using [https://sourceforge.net/p/forge/documentation/Shell%20Service/ Sourceforge Shell Service]:
# cd <directory above your htdocs directory, e.g., /home/user-web/vjohnson>
# svn checkout https://audacity.googlecode.com/svn/website/trunk/htdocs --username <your Google Code username>

=== Updating the site ===
This assumes you already have a working copy of the ''website'' SVN directory, and are in the top level htdocs folder.
# Set the '''SFUSER''' environment variable to your username on SourceForge, so the scripts know who to try and log in as. In the default bash shell, do something like ''export SFUSER=richardash1981'' (yes, Sourceforge - as the web host is still there, even though the version control isn't)
# The next step may overwrite changes to your working copy. You should have already committed those changes to SVN, in which case that does not matter, but if you want to check what will happen, run '''make pretend-pull''' first. 
# Run '''make pull''' to download files that aren't under SVN control from the web server into your working copy. 
# Bring your SVN working copy up to date without extraneous directories by running '''svn up'''.
# Check the output to ensure that all version controlled files are in sync with SVN. In general if they are not, then the SVN version is the correct one, so conflicting or modified files can be reverted to the SVN repository version (use '''svn revert''').  Files that are not in SVN should be left as-is, unless you know what you are doing / what they are.  They might be somebody's experimental files that the site relies on.
# If the .po files get into a mess, then you can delete all local changes to the .po files and get the SVN copies by doing '''make revert-po'''. This is just a short-cut to reverting them all with SVN.
# Run '''make pretend-push''' to see a list of altered files that will be uploaded to the SourceForge server (you will need the password for your SSH key (the one that communicates with Sourceforge web servers)). If this isn't what you expected, then check what is going now. No files are actually changed at this point.
# Run '''make'''. It will create first ''audacity_website.pot'', then update community/potdates.inc with the date of the updated website.pot file on the [http://audacity.sourceforge.net/community/translation Translators page], then update the .po files with the new strings, and finally compile all the .po files into .mo files. This can take a while and be quite CPU intensive.
# Run '''make pretend-push''' again. You should see that as well as the files you saw last time, locale/audacity_website.pot, community/potdates.inc and all the .mo files (in the individual language folders in locale) are scheduled to be uploaded to the server. No .po files will be uploaded because the rsync command ignores them (they are not needed on the web server).
# Finally run '''make push''' to upload the files.

== Adding new web site translations ==
This consists of three operations - merging strings with the new .po file to bring it up to date, compiling it to generate .mo files for the web site, and committing the new translation to SVN.

=== Prerequisites ===
The same tools and checked out copy of the ''website'' SVN directory are needed as for updating the site from SVN above.

=== Updating a translation ===
This assumes you already have a working copy of the ''website'' SVN directory, and are in the top level htdocs folder.
# Set the '''SFUSER''' environment variable to your username on SourceForge, so the scripts know who to try and log in as.
# Run '''make pull''' to download files that aren't under SVN control from the web server into your working copy. This may overwrite changes to your working copy, but that doesn't matter provided that you have committed them to SVN.
# Bring your SVN working copy up to date without extraneous directories by running '''svn up'''.
# Check the output to ensure that all version controlled files are in sync with SVN. In general if they are not, then the SVN version is the correct one, so conflicting or modified files can be reverted. It is very likely that .po files will have conflicts, and the SVN version should be used. A quick hack to revert all .po files is to run '''make revert-po'''.
# Run '''make pretend-push''' to see a list of altered files that will be uploaded to the SourceForge server (you will need the password for your SSH key). This should be an empty list, as no files have changed. If not, then something isn't right. No files are actually changed at this point.
# Overwrite the relevant language translation with the new .po file you have been sent by the translator.
# Run '''make''' to generate new translations based on the new file. Any strings that have been altered since the last update by the translator will be merged into the .po file.
# Run '''make pretend-push''' again. You should see that the relevant language's .mo file, and possibly locale/audacity_website.pot, is scheduled to be uploaded to the server. No .po files will be uploaded because the rsync command ignores them (they are not needed on the web server).
# Run '''make push''' to upload the files.
# Commit the updated .po file to SVN in the normal way.

== Line Endings and Cross-Platform Issues ==
The translation system will not work correctly if files with incorrect line endings end up on the (Linux) server where xgettext is run (by the Makefile) in order to create the .pot file. To prevent this, all files need to be stored in SVN with the ''svn:eol-style'' property set to ''native''. This property must be set once when each file is committed to SVN. Note that if you do an SVN copy (preserving the history) then the properties are also kept. If you do an ordinary file copy, they aren't kept when you do an Add.

To locate files which have not been correctly set, use this BASH command:
 for file in $(find . -name '*.php' -or -name '*.inc'); do style=$(svn propget svn:eol-style ${file});if [[ "${style}" != "native" ]]; then echo "${file}" ; fi ; done
These files need to be converted to the native line ending for the platform being worked on, the property set and the lot committed. To check the commit you will find the ''--ignore-eol-style'' sub-option useful:
 svn di -x --ignore-eol-style

You can prevent future problems by enabling the "auto-props" feature in Subversion's configuration file. See http://www.mediawiki.org/wiki/Subversion/auto-props for help setting appropriate [auto-props] entries.  

'''Hints for Tortoise SVN users on Windows:''' SVN properties can be added or edited by right-clicking over the file(s) > TortoiseSVN > Properties, or Properties then the "Subversion" tab of the Windows properties sheet. You can also edit the subversion config file on the "General" page of TortoiseSVN settings to set properties automatically on files and folders when they are added to the repository. To use native line endings on new files and folders, uncomment "enable-auto-props = yes" in config, then uncomment or add an appropriate entry (as above) in the [auto-props] section.  

It's a lot easier if this is got right before files are added!

[[Category: For Developers]]