Kalo Finnish Romani NLP Grammar

Finite state and Constraint Grammar based analysers, proofing tools and other resources

View the project on GitHub giellalt/lang-rmf

Page Content

  • src-fst-morphology-stems-adpositions.lexc.md
  • Romani kale adpositions
  • src-fst-morphology-stems-adverbs.lexc.md
  • Adverbs
  • src-fst-morphology-stems-conjunctions.lexc.md
  • conjunctions
  • src-fst-morphology-stems-determiners.lexc.md
  • Kale Romani determiners
  • src-fst-morphology-stems-interjections.lexc.md
  • Interjections
  • src-fst-morphology-stems-nouns.lexc.md
  • Noun stems
  • src-fst-morphology-stems-numerals.lexc.md
  • Kale Romani Numerals
  • src-fst-morphology-stems-particles.lexc.md
  • Particles
  • src-fst-morphology-stems-pronouns.lexc.md
  • Pronouns
  • src-fst-morphology-stems-rmf-propernouns.lexc.md
  • src-fst-morphology-stems-subjunctions.lexc.md
  • subjunctions
  • src-fst-morphology-stems-verbs.lexc.md
  • Verbs
  • src-fst-phonetics-txt2ipa.xfscript.md
  • src-fst-transcriptions-transcriptor-abbrevs2text.lexc.md
  • src-fst-transcriptions-transcriptor-numbers-digit2text.lexc.md
  • src-fst-transcriptions-transcriptor-symbols2text.lexc.md
  • tools-grammarcheckers-grammarchecker.cg3.md
  • DELIMITERS
  • TAGS AND SETS
  • Kalo Finnish Romani language model documentation

    All doc-comment documentation in one large file.


    src-cg3-disambiguator.cg3.md

    Kalo Finnish Romani disambiguator

    This is a file for automatic syntactic analysis of Kalo Finnish Romani. It is written within the Constraint grammar formalism. It is in an initial state, containing only a handful of rules.

    Tags and sets

    This section just defines all grammatical tags from the morphology.

    Rule section

    Adposition rules

    Rule Po: Pre or post? Removes Pr reading when Gen is to the left and not to the right.

    Adjective rules

    Rule Msc selects masculine adjective if following noun is masculine Rule Fem selects feminine adjective if following noun is feminine

    Pronoun rules

    Rule Pers removes Poss if next word is verb (to be refined)

    Determiner rules

    Rule DetSg removes Pl reading of Det if next word is singular noun.

    Rule DetSgNP removes Pl reading of Det if to the left is singular noun and only pre-np words inbetween

    Verb rules

    Rule Sg1 selects Sg1 if nominative me to the left.

    Rule Sg3 removes Pl3 reading for verb if word to the left is singular nominative N or Pron.


    This (part of) documentation was generated from src/cg3/disambiguator.cg3


    src-cg3-functions.cg3.md

    Kalo Finnish Romani syntactic functions.

    Nothing done with this topic so far. This file is just a dummy copy of the North Saami file.

    These sets model noun phrases (NPs). The idea is to first define whatever can occur in front of the head of the NP, and thereafter negate that with the expression WORD - premodifiers.

    The set NOT-NPMOD is used to find barriers between NPs. Typical usage: … (*1 N BARRIER NPT-NPMOD) … meaning: Scan to the first noun, ignoring anything that can be part of the noun phrase of that noun (i.e., “scan to the next NP head”)

    These were the set types.

    HABITIVE MAPPING

    sma object

    SUBJ MAPPING - leftovers

    OBJ MAPPING - leftovers

    HNOUN MAPPING


    This (part of) documentation was generated from src/cg3/functions.cg3


    src-fst-morphology-affixes-adjectives.lexc.md

    Adjective inflection

    Oblique cases not looked into.

    LEXICON BARO temaattiset o-loppuiset adjektiivit

    LEXICON BAROVALO temaattiset o-loppuiset adjektiivit, -alo, -ano-johtimella tehtyjä

    LEXICON BESKO atemaattiset o-loppuiset adjektiivit

    LEXICON FAARLIGO

    LEXICON DUUR

    LEXICON FROO

    LEXICON PRFPRS for participles

    LEXICON PRFPRS contains only -ide for now.

    LEXICON comp contains only -ilde for now, lexicon -ano, -alo


    This (part of) documentation was generated from src/fst/morphology/affixes/adjectives.lexc


    src-fst-morphology-affixes-nouns.lexc.md

    Noun inflection

    The inflection first gives N, Nom and gender tags, then point to oblique cases.

    The noun continuation lexica

    Masculines

    LEXICON RAKLO adds -o for Sg Nom and redirects to MSCOBL

    LEXICON LAU

    LEXICON BEREH

    LEXICON VUUDAR

    LEXICON GRAI

    LEXICON DZII

    LEXICON IBA

    LEXICON BEN

    LEXICON OS

    LEXICON NIS

    LEXICON RIS

    LEXICON TIS

    LEXICON SSIS

    LEXICON IS_PL

    LEXICON ES

    LEXICON US

    LEXICON YS

    Feminines

    LEXICON BAAR pheen, tunnuksettomat fem

    LEXICON KAALI

    LEXICON ANGRUSTI

    LEXICON KHURMIN

    LEXICON BORNIK

    LEXICON DAI

    LEXICON TSIMB

    LEXICON SKOOLA

    LEXICON RAADZA

    Morphology

    Masculine morphology

    LEXICON MSCOBL pointing to sg and pl lexica

    LEXICON MSCOBL_SG

    LEXICON MSCOBL_SG_ins_s

    LEXICON MSCOBL_PL

    LEXICON MSCOBL_PL_OBL

    LEXICON MSCOBL_PL_o

    Feminine morphology

    LEXICON FEMOBL pointing to sg and pl lexica

    LEXICON FEMOBL_SG

    LEXICON FEMOBL_PL

    LEXICON FEMOBL_a pointing to sg and pl lexica

    LEXICON FEMOBL_SG_a

    LEXICON FEMOBL_PL_a

    LEXICON FEMOBL_PL_OBL

    LEXICON FEMOBL_PL_o

    The oblique cases -ke, -ta, -ha

    LEXICON OBLCASES_SG

    LEXICON OBLCASES_SG_ins_s

    LEXICON OBLCASES_PL

    LEXICON GENCASES


    This (part of) documentation was generated from src/fst/morphology/affixes/nouns.lexc


    src-fst-morphology-affixes-prefixes.lexc.md

    Prefixes

    So far no prefixes in the model for Finnish Romany .

    Prefixes No prefixes so far. Are there any?


    This (part of) documentation was generated from src/fst/morphology/affixes/prefixes.lexc


    src-fst-morphology-affixes-pronouns.lexc.md

    Inflected pronominal forms

    Case inflection

    LEXICON me_pron minä

    LEXICON tu_pron “sinä”

    LEXICON jou_pron “hän”

    LEXICON es_case

    LEXICON SAARO

    LEXICON es_case_obl c

    LEXICON SAKKO

    LEXICON indef_temaattinen thematic adjectival declension. Oblique?


    This (part of) documentation was generated from src/fst/morphology/affixes/pronouns.lexc


    src-fst-morphology-affixes-propernouns.lexc.md

    Proper noun inflection

    Nothing done so far, just adding tags

    LEXICON prop-mal male names

    LEXICON prop-fem female names


    This (part of) documentation was generated from src/fst/morphology/affixes/propernouns.lexc


    src-fst-morphology-affixes-symbols.lexc.md

    Symbol affixes

    This content is just from the template, work has not begun.


    This (part of) documentation was generated from src/fst/morphology/affixes/symbols.lexc


    src-fst-morphology-affixes-verbs.lexc.md

    Kalo Finnish Romani verb inflection

    Under construction.

    First come auxiliaries, then lexical verb classes. Finally there is a section on common verb suffixes.

    Auxiliaries

    LEXICON HIN spells out the whole paradigm

    LEXICON VAA spells out the whole paradigm

    Lexical verbs

    The lexical setup follows the grammar, but several lexica may probably be unified.

    BOLL primary verb, other than r-stem

    TSEER primary verb, r-stem, d pret

    LEXICON TRAAD cns pres and id pret

    LEXICON TSUUV cns pres and nd pret

    LEXICON LURR cns pres and d pret

    LEXICON DZAMB cns pres and id pret

    LEXICON MANG cns pres and jom pret

    LEXICON DIKK cns pres and jom pret

    LEXICON KAMM cns pres and jom pret

    LEXICON BAHH cns pres and t pret

    LEXICON MEKK cns pres and l/n pret

    LEXICON D for “to do”, cns pres and jom pret

    LEXICON DZA vow pres and id pret

    LEXICON PILA cns pres and id pret “juoda”

    Transitive morphology

    LEXICON RAKKAVELA cns pres and ad pret + parallel forms

    LEXICON PHAGARVELA cns pres and ard pret

    LEXICON TSJALJEVELA cns pres and ed pret

    LEXICON TSEEKJIVELA id pret

    LEXICON DZORALVELA cns pres and ad pret

    Intransitive morphology

    LEXICON GARUVELA cns pres and jom pret

    LEXICON AGURVELA cns pres and jom pret

    Person suffixes

    Here come the person suffixes.

    LEXICON cns_pressuff cns pres: aa, eha, ela, …

    LEXICON vow_pressuff vcow pres: a, ha, la, …

    LEXICON pretsuff om, al, as…

    LEXICON pretsuff_jom jom, jal, lo, …


    This (part of) documentation was generated from src/fst/morphology/affixes/verbs.lexc


    src-fst-morphology-phonology.twolc.md

    The Romany morphophonological/twolc rules file

    Declarations and definitions

    Alphabet

    a b c d e f g h i j k l m n o p q r s t u v w x y z æ ø å á é ó ú í à è ò ù ì ä ë ö ü ï â ê ô û î ã ý þ ñ ð ß ç š ž ȟ

    A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Æ Ø Å Á É Ó Ú Í À È Ò Ù Ì Ä Ë Ö Ü Ï Â Ê Ô Û Î Ã Ý þ Ñ Ð Š Ž Ȟ

    %> %^V:0 vowel copy %^CDEL:0 trigger to delete consonant in front of suffix %^KK:0 archiphoneme for doubling k in kkiiro

    Sets

    Rules

    Consonant rules

    Vowel rules


    This (part of) documentation was generated from src/fst/morphology/phonology.twolc


    src-fst-morphology-root.lexc.md

    The Kalo Finnish Romany root.lexc file

    This file defines all multicharacter symbols, and contains the initial lexicon Root.

    Analysis symbols

    The morphological analyses of wordforms for the Romany language are presented in this system in terms of the following symbols. Note that some of the tags still are copied from other language versions

    Parts-of-speech tags:

    The number, gender and case tags:

    Other sub-tags

    Other tags

    Question and Focus particles (probably not needed)

    Semantic tags

    These are not yet in use

    Derivation

    Derivations are classified under the morphophonetic form of the suffix, the source and target part-of-speech.

    Usage tags

    The Usage extents are marked using the following tags:

    Morphophonology

    To represent phonologic variations in word forms we use the following symbols in the lexicon files:

    Flag diacritics

    We have manually optimised the structure of our lexicon using following flag diacritics to restrict morhpological combinatorics - only allow compounds with verbs if the verb is further derived into a noun again:

    The flag diacritics are not yet in use

    Flag Explanation
    @P.NeedNoun.ON@ (Dis)allow compounds with verbs unless nominalised
    @D.NeedNoun.ON@ (Dis)allow compounds with verbs unless nominalised
    @C.NeedNoun@ (Dis)allow compounds with verbs unless nominalised

    For languages that allow compounding, the following flag diacritics are needed to control position-based compounding restrictions for nominals. Their use is handled automatically if combined with +CmpN/xxx tags. If not used, they will do no harm.

    Flag Explanation
    @P.CmpFrst.FALSE@ Require that words tagged as such only appear first
    @D.CmpPref.TRUE@ Block such words from entering ENDLEX
    @P.CmpPref.FALSE@ Block these words from making further compounds
    @D.CmpLast.TRUE@ Block such words from entering R
    @D.CmpNone.TRUE@ Combines with the next tag to prohibit compounding
    @U.CmpNone.FALSE@ Combines with the prev tag to prohibit compounding
    @P.CmpOnly.TRUE@ Sets a flag to indicate that the word has passed R
    @D.CmpOnly.FALSE@ Disallow words coming directly from root.

    Use the following flag diacritics to control downcasing of derived proper nouns (e.g. Finnish Pariisi -> pariisilainen). See e.g. North Sámi for how to use these flags. There exists a ready-made regex that will do the actual down-casing given the proper use of these flags.

    Flag Explanation
    @U.Cap.Obl@ Allowing downcasing of derived names: deatnulasj.
    @U.Cap.Opt@ Allowing downcasing of derived names: deatnulasj.

    The Root lexicon

    LEXICON Root The word forms in Romany language start from the lexeme roots of basic word classes, or optionally from prefixes:

    LEXICON K (for clitics, probably not needed)


    This (part of) documentation was generated from src/fst/morphology/root.lexc


    src-fst-morphology-stems-adjectives.lexc.md

    Adjectives

    contains the adjectives, so far grouped by type. thematic = indo-aryan adjs, athematic = recent (post-1400) loans.

    LEXICON Adjectives

    (a) Temaattiset o-loppuiset adjektiivit

    (a) Atemaattiset o-loppuiset adjektiivit

    (c) Temaattiset johdetut adjektivit

    (d) Atemaattiset johdetut adjektivit

    Temaattiset ja atemaattiset tunnuksettomat adjektiivit

    Muut vokaaliloppuiset adjektiivit


    This (part of) documentation was generated from src/fst/morphology/stems/adjectives.lexc


    src-fst-morphology-stems-adpositions.lexc.md

    Romani kale adpositions

    Tags

    LEXICON pr for +Pr

    LEXICON po for +Po

    LEXICON ad for +Pr and +Po

    LEXICON Adpositions contains the list


    This (part of) documentation was generated from src/fst/morphology/stems/adpositions.lexc


    src-fst-morphology-stems-adverbs.lexc.md

    Adverbs

    LEXICON adv gives tag +Adv

    LEXICON Adverbs The list, 70 so far.


    This (part of) documentation was generated from src/fst/morphology/stems/adverbs.lexc


    src-fst-morphology-stems-conjunctions.lexc.md

    conjunctions

    LEXICON cc adds the tag +CC

    LEXICON Conjunctions contains 10 or so.


    This (part of) documentation was generated from src/fst/morphology/stems/conjunctions.lexc


    src-fst-morphology-stems-determiners.lexc.md

    Kale Romani determiners

    LEXICON Determiners 6 of them

    LEXICON DAUVA for dauva

    LEXICON KOUVA for kouva and douva


    This (part of) documentation was generated from src/fst/morphology/stems/determiners.lexc


    src-fst-morphology-stems-interjections.lexc.md

    Interjections

    LEXICON ij adds tag +Interj

    LEXICON Interjections contains one so far.


    This (part of) documentation was generated from src/fst/morphology/stems/interjections.lexc


    src-fst-morphology-stems-nouns.lexc.md

    Noun stems

    This documents the file stems/nouns.lexc. So far only close to 200 words distributed on different inflection types.

    Nominatiivi päättyy yksikössä mm. -o, -i, -os, -is, -es, -a ja monikossa mm. -a, -ja, -i. Instrumentaali saa yleensä yksikössäkin -sa-päätteen -iba- ja -ben- loppuisilla abstraktisubstantiiveilla: džaan-ibos-sa ’tiedolla’, ȟaaben- es-sa ’ruoan kanssa’ sekä sanoilla grai ’hevonen’ ja džii ’sydän’. Lyhyen genetiivin pääte on yksikössä -ko ja monikossa -go (rakl-es-ko ’pojan’, džuu-jen-go ’naisten’). Pitkä genetiivi päättyy yksikössä -kiir- o ja monikossa -giir-o (sikj-ibos-kiir-o ’opettaja’, tšell-ibon-giir-e ’tanssit’).9

    LEXICON Nouns

    maskuliinit

    -o, -os, -is, -es, us; -iba, -ben

    (a) o-vartalo: raklo

    (b) tunnuksettomia anguš, lau

    (c) i-loppuinen

    (c?) a-loppuinen

    (d) -iba/-irba/-ilba-loppuisia abstraktisubstantiiveja

    ben-loppuisia abstraktisubstantiiveja

    os-loppuisia

    is-loppuisia

    es-loppuisia

    us-loppuisia

    feminiinit

    i-loppuiset: butti, …

    tunnuksettomia: baar, …

    a-loppuisia: kamana, ..

    The long list

    To be added when basis is in place.


    This (part of) documentation was generated from src/fst/morphology/stems/nouns.lexc


    src-fst-morphology-stems-numerals.lexc.md

    Kale Romani Numerals

    Numerals so far only 1-10, the tens, hundred , thousand, million. No inflection added.

    LEXICON Numerals

    LEXICON Arabic shall be common files

    LEXICON Numbers preparing for composing

    LEXICON HIGHNUMERALS

    LEXICON TENS

    LEXICON 11to19 combining 11, 12, ..

    LEXICON 1to10

    LEXICON NUMCASE gets tag

    LEXICON MILJ


    This (part of) documentation was generated from src/fst/morphology/stems/numerals.lexc


    src-fst-morphology-stems-particles.lexc.md

    Particles

    In the grammar, Particles is a wide category, containing indeclineable words. Several of these should get syntactically defined POS.

    LEXICON pcle

    LEXICON Particles contains 30 or so words.


    This (part of) documentation was generated from src/fst/morphology/stems/particles.lexc


    src-fst-morphology-stems-pronouns.lexc.md

    Pronouns

    The pronouns are here. Some of the inflexion is in ../affixes.

    The lexicon Pronouns, splitting in subtypes

    LEXICON Pronouns with 5 sublexica

    Personal

    LEXICON Personal for me, jou, …

    Reflexive

    LEXICON Reflexive

    LEXICON reflpron

    Interrogative

    LEXICON Interrogative for koon…

    Indefinite

    LEXICON Indefinite

    Possessive

    LEXICON Possessive for mo, to, …


    This (part of) documentation was generated from src/fst/morphology/stems/pronouns.lexc


    src-fst-morphology-stems-rmf-propernouns.lexc.md

    LEXICON rmf-propernouns contains just a handful, with no inflection.


    This (part of) documentation was generated from src/fst/morphology/stems/rmf-propernouns.lexc


    src-fst-morphology-stems-subjunctions.lexc.md

    subjunctions

    LEXICON cs adds the tag +CS

    LEXICON Subjunctions for at, te, so, ka, …


    This (part of) documentation was generated from src/fst/morphology/stems/subjunctions.lexc


    src-fst-morphology-stems-verbs.lexc.md

    Verbs

    This lists verbs as presented in the grammar.

    LEXICON Verbs is the lexicon, containing the verb list

    Auxiliaries

    som+V: HINVAA ; copula hin, tulla vaa

    Lexical verbs

    Consonant stems

    (a) Other than r-stems, primary verbs

    6.23 (b) d-preteriti, stem -l-

    6.23 (b) d-preteriti, stem -v-

    (b) r-stems, primary verbs

    6.23 (a) d-preteriti, stem -r-

    id past tense

    6.24 stem b, d, t, id past tense

    6.24 stem -n-, past tens -id

    6.26 t past tense

    6.27 Stem -g, -k, past tense l

    Transitive derivatives
    Intransitive derivatives

    Vowel stem, primary verbs


    This (part of) documentation was generated from src/fst/morphology/stems/verbs.lexc


    src-fst-phonetics-txt2ipa.xfscript.md

    retroflex plosive, voiceless t ʈ 0288, 648 ( = ASCII 096) retroflex plosive, voiced d ɖ 0256, 598 labiodental nasal F ɱ 0271, 625 retroflex nasal n ɳ 0273, 627 palatal nasal J ɲ 0272, 626 velar nasal N ŋ 014B, 331 uvular nasal N\ ɴ 0274, 628

    bilabial trill B\ ʙ 0299, 665 uvular trill R\ ʀ 0280, 640 alveolar tap 4 ɾ 027E, 638 retroflex flap r ɽ 027D, 637 bilabial fricative, voiceless p\ ɸ 0278, 632 bilabial fricative, voiced B β 03B2, 946 dental fricative, voiceless T θ 03B8, 952 dental fricative, voiced D ð 00F0, 240 postalveolar fricative, voiceless S ʃ 0283, 643 postalveolar fricative, voiced Z ʒ 0292, 658 retroflex fricative, voiceless s ʂ 0282, 642 retroflex fricative, voiced z` ʐ 0290, 656 palatal fricative, voiceless C ç 00E7, 231 palatal fricative, voiced j\ ʝ 029D, 669 velar fricative, voiced G ɣ 0263, 611 uvular fricative, voiceless X χ 03C7, 967 uvular fricative, voiced R ʁ 0281, 641 pharyngeal fricative, voiceless X\ ħ 0127, 295 pharyngeal fricative, voiced ?\ ʕ 0295, 661 glottal fricative, voiced h\ ɦ 0266, 614

    alveolar lateral fricative, vl. K alveolar lateral fricative, vd. K\

    labiodental approximant P (or v) alveolar approximant r\ retroflex approximant r` velar approximant M\

    retroflex lateral approximant l` palatal lateral approximant L velar lateral approximant L
    Clicks

    bilabial O\ (O = capital letter) dental |
    (post)alveolar !\ palatoalveolar =\ alveolar lateral ||
    Ejectives, implosives

    ejective > e.g. ejective p p> implosive < e.g. implosive b b< Vowels

    close back unrounded M close central unrounded 1 close central rounded } lax i I lax y Y lax u U

    close-mid front rounded 2 close-mid central unrounded @\ close-mid central rounded 8 close-mid back unrounded 7

    schwa ə @

    open-mid front unrounded E open-mid front rounded 9 open-mid central unrounded 3 open-mid central rounded 3\ open-mid back unrounded V open-mid back rounded O

    ash (ae digraph) { open schwa (turned a) 6

    open front rounded & open back unrounded A open back rounded Q Other symbols

    voiceless labial-velar fricative W voiced labial-palatal approx. H voiceless epiglottal fricative H\ voiced epiglottal fricative <\ epiglottal plosive >\

    alveolo-palatal fricative, vl. s\ alveolo-palatal fricative, voiced z\ alveolar lateral flap l\ simultaneous S and x x\ tie bar _ Suprasegmentals

    primary stress “ secondary stress % long : half-long :\ extra-short _X linking mark -
    Tones and word accents

    level extra high _T level high _H level mid _M level low _L level extra low _B downstep ! upstep ^ (caret, circumflex)

    contour, rising contour, falling _F contour, high rising _H_T contour, low rising _B_L

    contour, rising-falling _R_F (NB Instead of being written as diacritics with _, all prosodic marks can alternatively be placed in a separate tier, set off by < >, as recommended for the next two symbols.) global rise global fall Diacritics

    voiceless 0 (0 = figure), e.g. n_0 voiced _v aspirated _h more rounded _O (O = letter) less rounded _c advanced _+ retracted _- centralized _” syllabic = (or _=) e.g. n= (or n=) non-syllabic _^ rhoticity `

    breathy voiced _t creaky voiced _k linguolabial _N labialized _w palatalized ‘ (or _j) e.g. t’ (or t_j) velarized _G pharyngealized _?\

    dental d apical _a laminal _m nasalized ~ (or _~) e.g. A~ (or A~) nasal release _n lateral release _l no audible release _}

    velarized or pharyngealized _e velarized l, alternatively 5 raised _r lowered _o advanced tongue root _A retracted tongue root _q


    This (part of) documentation was generated from src/fst/phonetics/txt2ipa.xfscript


    src-fst-transcriptions-transcriptor-abbrevs2text.lexc.md

    We describe here how abbreviations are in Kalo Finnish Romani are read out, e.g. for text-to-speech systems.

    For example:


    This (part of) documentation was generated from src/fst/transcriptions/transcriptor-abbrevs2text.lexc


    src-fst-transcriptions-transcriptor-numbers-digit2text.lexc.md

    % komma% :, Root ; % tjuohkkis% :%. Root ; % kolon% :%: Root ; % sárggis% :%- Root ; % násti% :%* Root ;


    This (part of) documentation was generated from src/fst/transcriptions/transcriptor-numbers-digit2text.lexc


    src-fst-transcriptions-transcriptor-symbols2text.lexc.md

    This file contains mappings from abbreviations and some acronyms to full forms for text-to-speech purposes. This is a supplement to the analyser; the analyser must tag the strings as +ABBR or similar for the transcriptions to work. The resulting full form must be lemmas known to the analyser, for further processing.

    We describe here how abbreviations in Kalo Finnish Romani are read out, for text-to-speech systems.

    The file contains:


    This (part of) documentation was generated from src/fst/transcriptions/transcriptor-symbols2text.lexc


    tools-grammarcheckers-grammarchecker.cg3.md

    [ L A N G U A G E ] G R A M M A R C H E C K E R

    DELIMITERS

    TAGS AND SETS

    Tags

    This section lists all the tags inherited from the fst, and used as tags in the syntactic analysis. The next section, Sets, contains sets defined on the basis of the tags listed here, those set names are not visible in the output.

    Beginning and end of sentence

    BOS EOS

    Parts of speech tags

    N A Adv V Pron CS CC CC-CS Po Pr Pcle Num Interj ABBR ACR CLB LEFT RIGHT WEB PPUNCT PUNCT

    COMMA ¶

    Tags for POS sub-categories

    Pers Dem Interr Indef Recipr Refl Rel Coll NomAg Prop Allegro Arab Romertall

    Tags for morphosyntactic properties

    Nom Acc Gen Ill Loc Com Ess Ess Sg Du Pl Cmp/SplitR Cmp/SgNom Cmp/SgGen Cmp/SgGen PxSg1 PxSg2 PxSg3 PxDu1 PxDu2 PxDu3 PxPl1 PxPl2 PxPl3 Px

    Comp Superl Attr Ord Qst IV TV Prt Prs Ind Pot Cond Imprt ImprtII Sg1 Sg2 Sg3 Du1 Du2 Du3 Pl1 Pl2 Pl3 Inf ConNeg Neg PrfPrc VGen PrsPrc Ger Sup Actio VAbess

    Err/Orth

    Semantic tags

    Sem/Act Sem/Ani Sem/Atr Sem/Body Sem/Clth Sem/Domain Sem/Feat-phys Sem/Fem Sem/Group Sem/Lang Sem/Mal Sem/Measr Sem/Money Sem/Obj Sem/Obj-el Sem/Org Sem/Perc-emo Sem/Plc Sem/Sign Sem/State-sick Sem/Sur Sem/Time Sem/Txt

    HUMAN

    PROP-ATTR PROP-SUR

    TIME-N-SET

    Syntactic tags

    @+FAUXV @+FMAINV @-FAUXV @-FMAINV @-FSUBJ> @-F<OBJ @-FOBJ> @-FSPRED<OBJ @-F<ADVL @-FADVL> @-F<SPRED @-F<OPRED @-FSPRED> @-FOPRED> @>ADVL @ADVL< @<ADVL @ADVL> @ADVL @HAB> @<HAB @>N @Interj @N< @>A @P< @>P @HNOUN @INTERJ @>Num @Pron< @>Pron @Num< @OBJ @<OBJ @OBJ> @OPRED @<OPRED @OPRED> @PCLE @COMP-CS< @SPRED @<SPRED @SPRED> @SUBJ @<SUBJ @SUBJ> SUBJ SPRED OPRED @PPRED @APP @APP-N< @APP-Pron< @APP>Pron @APP-Num< @APP-ADVL< @VOC @CVP @CNP OBJ

    -OTHERS SYN-V @X ### Sets containing sets of lists and tags This part of the file lists a large number of sets based partly upon the tags defined above, and partly upon lexemes drawn from the lexicon. See the sourcefile itself to inspect the sets, what follows here is an overview of the set types. #### Sets for Single-word sets INITIAL #### Sets for word or not WORD NOT-COMMA #### Case sets ADLVCASE CASE-AGREEMENT CASE NOT-NOM NOT-GEN NOT-ACC #### Verb sets NOT-V #### Sets for finiteness and mood REAL-NEG MOOD-V NOT-PRFPRC #### Sets for person SG1-V SG2-V SG3-V DU1-V DU2-V DU3-V PL1-V PL2-V PL3-V #### Pronoun sets #### Adjectival sets and their complements #### Adverbial sets and their complements #### Sets of elements with common syntactic behaviour #### NP sets defined according to their morphosyntactic features #### The PRE-NP-HEAD family of sets These sets model noun phrases (NPs). The idea is to first define whatever can occur in front of the head of the NP, and thereafter negate that with the expression **WORD - premodifiers**. #### Border sets and their complements #### Grammarchecker sets * * * This (part of) documentation was generated from [tools/grammarcheckers/grammarchecker.cg3](https://github.com/giellalt/lang-rmf/blob/main/tools/grammarcheckers/grammarchecker.cg3) --- ## tools-tokenisers-tokeniser-disamb-gt-desc.pmscript.md ## Tokeniser for rmf Usage: ``` $ make $ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://github.com/hfst/hfst/wiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1. unknown word-like forms, and 2. unmatched strings We want to give 1) a match, but let 2) be treated specially by `hfst-tokenise -a` Unknowns are made of: * lower-case ASCII * upper-case ASCII * select extended latin symbols ASCII digits * select symbols * Combining diacritics as individual symbols, * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" ### Unknown handling Unknowns are tagged ?? and treated specially with `hfst-tokenise` hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Finally we mark as a token any sequence making up a: * known word in context * unknown (OOV) token in context * sequence of word and punctuation * URL in context * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-disamb-gt-desc.pmscript](https://github.com/giellalt/lang-rmf/blob/main/tools/tokenisers/tokeniser-disamb-gt-desc.pmscript) --- ## tools-tokenisers-tokeniser-gramcheck-gt-desc.pmscript.md ## Grammar checker tokenisation for rmf Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc) Then just: ``` $ make $ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` More usage examples: ``` $ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://github.com/hfst/hfst/wiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1) unknown word-like forms, and 2) unmatched strings We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a * select extended latin symbols * select symbols * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" TODO: Could use something like this, but built-in's don't include šžđčŋ: Simply give an empty reading when something is unknown: hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Finally we mark as a token any sequence making up a: * known word in context * unknown (OOV) token in context * sequence of word and punctuation * URL in context * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript](https://github.com/giellalt/lang-rmf/blob/main/tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript) --- ## tools-tokenisers-tokeniser-tts-cggt-desc.pmscript.md ## TTS tokenisation for smj Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc) Then just: ```sh make echo "ja, ja" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` More usage examples: ```sh echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa \ boasttu olmmoš, man mielde lahtuid." \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst echo "márffibiillagáffe" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://kitwiki.csc.fi/twiki/bin/view/KitWiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1) unknown word-like forms, and 2) unmatched strings We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a * select extended latin symbols * select symbols * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" TODO: Could use something like this, but built-in's don't include šžđčŋ: Simply give an empty reading when something is unknown: hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Needs hfst-tokenise to output things differently depending on the tag they get * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-tts-cggt-desc.pmscript](https://github.com/giellalt/lang-rmf/blob/main/tools/tokenisers/tokeniser-tts-cggt-desc.pmscript)

    Sitemap