GiellaLT provides rule-based language technology aimed at minority and indigenous languages
This is an overview of the file structure for each language catalogue found in the GiellaLT infrastructure, i.e. over the repositories giella-core and each of the lang-$ISO repositories.
Each language has its own repository, named with a prefix
lang- followed by the
ISO 639-3 code, and possibly
extended according to BCP47 if needed.
Each language repository contains the following directories:
docs(folder where we write language-specific documentation)
m4(support files for compilation)
misc(a grab bag dir for anything you don’t want in svn - all files are ignored)
src(this is where the linguistic source files are)
hyphenation(each phenomenon its folder)
orthography(capital letters, spellrelax)
phonetics(for different scripts for phonetic transcription)
tools(tools we build, both proofing tools and other tools)
Here’s a tree view of the structure, shown for the undefined language
(and thus the dir structure of all languages):
und$ tree -d . ├── devtools ├── docs ├── m4 ├── misc ├── src │ ├── cg3 │ ├── filters │ ├── fst │ │ ├── affixes │ │ └── stems │ ├── hyphenation │ ├── orthography │ ├── phonetics │ ├── tagsets │ └── transcriptions ├── test │ ├── data │ ├── src │ │ ├── morphology │ │ ├── phonology │ │ └── syntax │ └── tools │ ├── hyphenators │ ├── mt │ │ └── apertium │ └── spellcheckers └── tools ├── analysers ├── grammarcheckers ├── hyphernators ├── mt │ ├── apertium │ └── cgbased ├── shellscripts ├── spellcheckers └── tokenisers
Some directories are described in further detail below.
root.lexc(defines tags and basic parts of speech)
affixes/(here we find the lexc morphology files)
stems/(lexc stem files, the lists of words)
fst/ folder itself might also contain some lexc files, like:
The makefile defines two, perhaps four variables (the two first must be defined)
We define all the source files we need to build the transducers. The build system will take care of putting them together and compiling them.
This statement includes the majority of the build instructions. You should never need to touch the included file.
Here we take care of initial capitalisation, of systematic spelling relaxation, and of alternative orthographies, if needed. Note that spellrelax now is language-specific.
This folder contains files for conversion to IPA.
This folder contains
functions.cg3, dependency.cg3 for sma, sme, smj and smn are in
giella-shared/smi/src/cg3/. Faroese also uses the common
dependency.cg3, but has its own
Within it there are several subdirs for different tools to test. Each test is wrapped in a shell script that emits one of the following values, depending on the outcome of the test:
If you need a new test, just write a shell script that follows this convention, and add it to the TESTS variable in the Makefile.am file. That’s it.
Also take care that the shell script uses AM variables for all references to files and commands outside the script which can not be assumed to be universally available. This will make the test scripts portable.
What kind of and how many transducers we produce varies from language to language.
The binary files are built in
the naming conventions described here.
For some languages (at least sme and sma) the content of and difference between the transducers is explained on the documentation page of each language
giella-shared are dependents of all other repositories.
The former is for
language-independent technical files and build instructions, and the latter is for
language-independent filters, and linguistic resources shared among many or all languages.
Both are automatically checked out (svn) or cloned (git) when initially configuring a new language.
giella-core contains scripts used for all languages.