GiellaLT provides an infrastructure for rule-based language technology aimed at minority and indigenous languages, and streamlines building anything from keyboards to speech technology. Read more about Why. See also How to get started and our Privacy document.
First we will look at typical maintenance tasks, and more detailed tasks will follow.
Here is how you can expect to work (be in neahtta):
The nds script contains shortcuts for running all the following processes.
First ensure you are logged in as the user neahtta and are in the project directory.
cd ~/neahtta/
Then, update the new translations you already checked in locally:
git pull
Then, run the compile command
nds strings compile
The following command will compile new strings to be localised based upon updates in the different templates, and may result in some errors if a locale is not installed on the system. If you do not need these locales, this is not a problem, and you should use the project-specific command above. If you need these locales, this is a problem, and see below on the section on Adding new locales.
nds strings compile
Sometimes you (or other maintainers) want to revise or enlarge the dictionary interface. This should be done in the underlying html files, e.g. here:
If in doubt, be careful, read documentation on a relevant django page.
If the source code has changed, this has likely resulted in line numbers being out of date. Thus, you will need to re-extract the strings. Babel is pretty smart about doing this.
First ensure you are in the project directory.
cd ~/neahtta/
Then, extract strings:
nds strings extract
cd ~/neahtta/nds strings compileIf everything went good and there were no errors, continue, otherwise if you need help troubleshooting see the section below (Common error messages and troubleshooting).
Then, you need to restart the relevant services.
nds restart PROJNAME
(Where PROJNAME is a standin for the project name, e.g., baakoeh, valks, sanit, etc.)
To write localisation (translate the interface to different languages) you have to find the place of the specific file to localise
neahtta>find . -name "templates" -type d
./configs/language_specific_rules/templates
./templates
Then you go to the specific directory, for instance, here:
configs/language_specific_rules/templates/
There you find your directory (follow your ISO code). Edit the lines labeled msgstr in the file LC_MESSAGES/messages.po. Note from the examples the way the quotation marks are put, with errors here the file will not work. Do not change the text on the lines marked #: or msgid.
Then check in your changes. To update the server, follow the procedure in the section Recompiling translations for production server below.
The form of the msgid is not important (it is translated), but an msgid with wrong or irrelevant content may lead the translators astray.
Over each msgid there is a file reference, go there and correct the original text. It is always found with a tag
{ trans %}
before and a tag
{ endtrans %}
after the string to be translated. You thus change what you find between those two tags.
Then, you should compile new translations.po files. This is done on the NDS server. Go there, and do
nds strings compile
Afterwards, check in the new .po files in translations/*/LC_MESSAGES/messages.po
Also read the Oahpa documentation on localisation in order to see how this works for Oahpa.
Internationalization (i18n) in Neahttadigisánit uses Flask-Babel which is an interface for flask to the Python Babel extension
Translation strings are marked as they were in the Oahpa programs, but NDS uses a slightly different template engine, there are however no major differences.
Marking translation strings in templates: Jinja documentation
In order to run these commands locally on your own machine, you will need to have a virtualenv installed. This is because there are a few packages required to extract strings, and it’s easier to just use the automatic install script.
See: Developing NDS
This comes in three main steps:
On UiT’s gtdict server the environment is already configured with the required tools. If you wish to do this on your local machine, you will need to set up your environment before this works.
For documentation on this, see: Developing NDS
All further steps assume that you are doing this on gtdict, if you are developing on your own machine, you will need to know what paths you need to update.
The system knows some locales, but not all. Unknown locales such as crk and sma need to be defined somewhere. North Saami is in existence in se.dat, se_NO.dat and se_FI.dat, but this does not help with languages without two-letter codes. Thus, for those we need to copy some files.
The required files must be copied from the NDS localedata ddirectory into babel’s locale-data directory. This should be done automatically by NDS if encountered while running (see i18n/utils.py:copy_custom_locales), but if it is not, you may have to do it manually. The files should be put in neahttadigisanit/src/env/lib/python3.9/site-packages/babel/locale-data/sma.dat (or another file for another language). All existing custom localization files are provided for convenience in the repository. If the one you need is not there, then you need to generate it yourself. See instructions at NDSLocalisations.
Thus the new locale will be able to be initialized with the init command below.
NB: If we were feeling more specific in this step, we could create a new locale specifically for the language, but since specifics aren’t required here it is fine to do it this way.
First make sure that all the message files have been updated to the most recent version of the code, by extracting all strings (above).
Now you can initialize a translation.
pybabel init -i translations/messages.pot -d translations -l LANG
Where LANG is the ISO code for the language project.
If you do not see the environment enabled in the command prompt when you log in as the Neahttadigisánit user, run the following from this user’s home directory:
source venv/bin/activate
If you do not see changes to the command prompt name, check which python to make sure it is active if you do not see it in the bash prompt. If it still doesn’t work, there are probably problems with virtualenv.
Typically you should see something like:
(venv)[neahtta@gtdict ~]$
Because most browsers report what language they use using the 2-character ISO 639-1 code, any localization that has a 2-character code, (north sámi has se, so lucky!), we must use this code. If it doesn’t, then creating new localizations with the 3-character code (ISO 639-2) will be necessary. Also, neahtta.py may not actually handle some of these 3-character localizations as well, so this will need to be fixed.
May be found in translations/, but for ease of finding out:
This means that the locale directory is missing a .dat file for the locale. Follow the procedure in the ‘Adding new locales’ section.
Babel marks things with #, fuzzy when it can’t find the line in the source code that the string belongs on, however it’s not a big problem if line numbers are missing. As such, to compile, delete lines containing #, fuzzy, and alternatively determine why there are no line numbers.
Sometimes the first translation string in the catalogue will be marked fuzzy, but this is because it is blank. The first message must be blank for the meta-data, so delete the fuzzy comment if it appears, but leave the string.