Khanty language model documentation
All doc-comment documentation in one large file.
src-cg3-disambiguator.cg3.md
K H A N T Y D I S A M B I G U A T O R
Nore! Some tag declarations or sets may be left from copying this file from sma.
@CODE
- LIST N = N ;
- LIST Sg = Sg ;
- LIST Pl = Pl ;
- LIST Nom = Nom ;
- LIST Acc = Acc ;
- LIST Gen = Gen ;
- LIST Par = Par ;
- LIST Loc = Loc ;
- LIST Abl = Abl ;
- LIST Lat = Lat ;
- LIST Tra = Tra ;
- LIST PxSg1 = PxSg1 ;
- LIST PxSg2 = PxSg2 ;
- LIST PxSg3 = PxSg3 ;
- LIST PxDu1 = PxDu1 ;
- LIST PxDu2 = PxDu2 ;
- LIST PxDu3 = PxDu3 ;
- LIST PxPl1 = PxPl1 ;
- LIST PxPl2 = PxPl2 ;
- LIST PxPl3 = PxPl3 ;
- LIST V = V ;
- LIST Prs = Prs ;
- LIST Prt = Prt ;
- LIST Sg1 = Sg1 ;
- LIST Sg2 = Sg2 ;
- LIST Sg3 = Sg3 ;
- LIST Du1 = Du1 ;
- LIST Du2 = Du2 ;
- LIST Du3 = Du3 ;
- LIST Pl1 = Pl1 ;
- LIST Pl2 = Pl2 ;
- LIST Pl3 = Pl3 ;
- LIST Inf = Inf ;
- LIST Neg = Neg ;
- LIST ConNeg = ConNeg ;
- LIST Foc/gan = Foc/gan ;
- LIST PrfPrc = PrfPrc ;
- LIST PrsPrc = PrsPrc ;
- LIST VGen = VGen ;
- LIST Ger = Ger ;
- LIST Ind = Ind ;
- LIST Imp = Imp ;
- LIST ImpII = ImpII ;
- LIST Pot = Pot ;
- LIST Cond = Cond ;
- LIST Imprt = Imprt ;
- LIST IV = IV ;
- LIST TV = TV ;
- LIST Opt = Opt ;
- LIST Actor = Actor ;
- LIST Clt = Clt ;
- LIST A = A ;
- LIST Pos = Pos ;
- LIST Ord = Ord ;
- LIST Coll = Coll ;
- LIST ABBR = ABBR ;
- LIST ACR = ACR ;
- LIST Comp = Comp ;
- LIST Superl = Superl ;
- LIST Attr = Attr ;
- LIST Pron = Pron ;
- LIST Pers = Pers ;
- LIST Dem = Dem ;
- LIST Interr = Interr ;
- LIST Rel = Rel ;
- LIST Po = Po ;
- LIST Pr = Pr ;
- LIST Adv = Adv ;
- LIST Interj = Interj ;
- LIST Indef = Indef ;
- LIST Num = Num ;
- LIST Prop = Prop ;
- LIST Mal = Mal ;
- LIST Fem = Fem ;
- LIST Sur = Sur ;
- LIST Org = Org ;
- LIST CS = CS ;
- LIST CC = CC ;
- LIST Pcle = Pcle ;
- LIST CLB = CLB ;
- LIST LEFT = LEFT ;
- LIST RIGHT = RIGHT ;
- LIST COMMA = “,” ;
- LIST Der1 = Der1 ;
- LIST Der2 = Der2 ;
- LIST Der3 = Der3 ;
- LIST Sem/Ani = Sem/Ani ;
- LIST Sem/Fem = Sem/Fem ;
- LIST Sem/Mal = Sem/Mal ;
- LIST Sem/Obj = Sem/Obj ;
- LIST Sem/Org = Sem/Org ;
- LIST Sem/Plc = Sem/Plc ;
- LIST Sem/Sur = Sem/Sur ;
- LIST Sem/Time = Sem/Time ;
- LIST Sem/Hum = Sem/Hum ;
- LIST Sem/Date = Sem/Date ;
- LIST Sem/Year = Sem/Year ;
- LIST Sem/Group = Sem/Group ;
- LIST Sem/Route = Sem/Route ;
- LIST Sem/Build = Sem/Build ;
- LIST Sem/Place = Sem/Place ;
- LIST Sem/Food = Sem/Food ;
- SET FIRSTNAME = (Prop Sem/Fem) OR (Prop Sem/Mal) ;
- LIST @CNP = @CNP ;
- LIST @CVP = @CVP ;
- LIST @+FAUXV = @+FAUXV ;
- LIST @+FMAINV = @+FMAINV ;
- LIST @-FAUXV = @-FAUXV ;
- LIST @-FMAINV = @-FMAINV ;
- SET MAINV = @-FMAINV OR @+FMAINV ;
- LIST @ADVL = @ADVL ;
- LIST @>ADVL = @>ADVL ;
- LIST @ADVL< = @ADVL< ;
- LIST @<ADVL = @<ADVL ;
- LIST @ADVL> = @ADVL> ;
- LIST @-FADVL = @-FADVL ;
- LIST @A< = @A< ;
- LIST @>A = @>A ;
- LIST @ActioN> = @ActioN> ;
- LIST @APP = @APP ;
- LIST @APP-N< = @APP-N< ;
- LIST @APP-Pron< = @APP-Pron< ;
- LIST @APP>Pron = @APP>Pron ;
- LIST @APP-Num< = @APP-Num< ;
- LIST @APP-ADVL< = @APP-ADVL< ;
- LIST @CMPND = @CMPND ;
- LIST @COMP-CS< = @COMP-CS< ;
- LIST @HAB = @HAB ;
- LIST @INTERJ = @INTERJ ;
- LIST @MEASURE = @MEASURE ;
- LIST @>N = @>N ;
- LIST @N< = @N< ;
- LIST @NNum> = @NNum>;
- LIST @NumN< = @NumN<;
- LIST @>Num = @>Num;
- LIST @Num< = @Num< ;
- LIST @NPron< = @NPron< ;
- LIST @NQ< = @NQ< ;
- LIST @NUM-PRON = @NUM-PRON ;
- LIST @OBJ = @OBJ ;
- LIST @<OBJ = @<OBJ ;
- LIST @OBJ> = @OBJ> ;
- LIST @OPRED = @OPRED ;
- LIST @<OPRED = @<OPRED ;
- LIST @OPRED> = @OPRED> ;
- LIST @PCLE = @PCLE ;
- LIST @HNOUN = @HNOUN ;
- LIST @PrcN> = @PrcN> ;
- LIST @PronN< = @PronN< ;
- LIST @PronN> = @PronN> ;
- LIST @Pron< = @Pron< ;
- LIST @>Pron = @>Pron ;
- LIST @P< = @P< ;
- LIST @>P = @>P ;
- LIST @SPRED = @SPRED ;
- LIST @<SPRED = @<SPRED ;
- LIST @SPRED> = @SPRED> ;
- LIST @SUBJ = @SUBJ ;
- LIST @<SUBJ = @<SUBJ ;
- LIST @SUBJ> = @SUBJ> ;
- LIST @SUBJ-QH = @SUBJ-QH ;
- LIST @TITLE = @TITLE ;
- LIST @VOC = @VOC ;
- LIST @X = @X ;
Sets
Grammatical sets
- LIST CASES = Nom Acc Gen Par Loc Abl Lat Tra ;
- LIST ADVLCASE = Loc Abl Lat Tra ;
- LIST NUMBER = Sg Du Pl ;
- LIST WORD = N A V Adv Pron Interj Num CS CC Pcle Pr Po ;
- SET REALWORD = WORD - Num - Ord ;
- SET REALWORD-NOTABBR = WORD - Num - Ord - ABBR ;
Sets for NP identification
- LIST NOT-ADV-INT-PCLE = N A Num Pron A V CC CS ;
- SET NOT-ADV = WORD - Adv ;
- SET NP-HEAD = Num OR N OR Pron ;
- SET PRE-NP-HEAD = (Prop Attr) OR (Prop @>N) OR (A Attr) OR (ABBR Attr) OR (Pron Pers Gen) OR (Pron Logo Gen) OR (N Gen) OR Num OR (Cmpnd) OR CC OR (Pron Dem) OR (Pron Refl Gen) OR (Pron Indef) OR (PrfPrc @>N) OR (PrfPrc @>N) OR (PrsPrc) OR (A Ord) OR Attr ;
- SET NP-MEMBER = PRE-NP-HEAD OR N ;
- SET NPNH = WORD - PRE-NP-HEAD OR (ABBR) ; NPNH = “NOT-PRE-NP-HEAD”
- SET NPNHA = WORD - PRE-NP-HEAD OR (ABBR) OR Adv ;
- SET NPNHAI = WORD - PRE-NP-HEAD - Adv - Indef ;
Noun sets
- SET PX = PxSg1 OR PxSg2 OR PxSg3 OR PxDu1 OR PxDu2 OR PxDu3 OR PxPl1 OR PxPl2 OR PxPl3 ;
Verb sets
The set REALCOPULAS is smaller than COPULAS, made for verbs with PrfPrc complements: Seammás REALCOPULAS son dovdan iežas…
- LIST REALCOPULAS = “lea” ;
The set COPULAS is for predicative constructions
- LIST COPULAS = REALCOPULAS ;
- SET V-NOT-COP = V - COPULAS ;
Verbs that never have arguments of their own
These verbs can take arguments, so they do not belong in the AUX group,
but they are nevertheless mapped to (@+FAUXV).
- SET AUX = COPULAS OR Neg ;
- SET VFIN = Ind OR Imprt ;
- SET VFIN-NOT-AUX = VFIN - AUX ;
- SET V-MAIN = V - COPULAS - Neg ;
- SET VFIN-NOT-NEG = VFIN - Neg ;
- SET NOT-NP = VFIN-NOT-NEG OR ConNeg OR Inf OR Pcle OR Interj OR CS ;
The set NP-BOUNDARY is bigger.
Boundary sets
Empty for now.
Disambiguation rules
BEFORE-SECTIONS
Rule: Date1 for adding Sem/Date as a tag to readings which looks like dates.
Rule: Date2 for adding Sem/Date as a tag to readings which looks like dates.
Rule: Date3 for adding Sem/Date as a tag to readings which looks like dates.
Rule: Date4 for adding Sem/Date as a tag to readings which looks like dates.
SECTION
Cycle 0: No context around the target word
Still no rules written.
Cycle 1: Local context around the target word
Rule: Attr removes Pos and keeps Attr for A in front of A or N.
Rule: Pos removes Attr and keeps Pos for A if no .
Cycle 2: Slightly less local context
Still no rules written.
Cycle 3: Global disambiguation
Still no rules written.
Cycle 4: Syntactic disambiguation
Still no rules written.
Cycle 5: Post-syntactic morphological disambiguation
Still no rules written.
This (part of) documentation was generated from src/cg3/disambiguator.cg3
src-cg3-functions.cg3.md
-
Sets for POS sub-categories
-
Sets for Semantic tags
-
Sets for Morphosyntactic properties
-
Sets for verbs
-
V is all readings with a V tag in them, REAL-V should
be the ones without an N tag following the V.
The REAL-V set thus awaits a fix to the preprocess V … N bug.
-
The set COPULAS is for predicative constructions
-
NP sets defined according to their morphosyntactic features
-
The PRE-NP-HEAD family of sets
These sets model noun phrases (NPs). The idea is to first define whatever can
occur in front of the head of the NP, and thereafter negate that with the
expression WORD - premodifiers.
The set NOT-NPMOD is used to find barriers between NPs.
Typical usage: … (*1 N BARRIER NPT-NPMOD) …
meaning: Scan to the first noun, ignoring anything that can be
part of the noun phrase of that noun (i.e., “scan to the next NP head”)
These were the set types.
HABITIVE MAPPING
-
hab1
-
hab2
-
hab3 ( @ADVL>) for hab-actor and hab-case; if leat to the right, and Nom to the right of leat. Lots of restrictions.
-
habNomLeft
-
hab4
-
hab6
-
hab7
- hab8 This is not HAB
-
hab5 This is not HAB
-
habDain ( @ADVL>) for (Pron Dem Pl Loc) if leat followed by Nom to the right
-
habGen ( @<ADVL) hab for Gen; if Gen is located in the end of the sentence and Nom is sentence initial
-
spred<obj (@SPRED<OBJ) for Acc; the object of an SPRPED. Not to be mistaken with OPRED. If SPRED is to the left, and copulas is to the left of it. Nom or Hab are found sentence initially.
-
Hab<spred (@<SPRED) for Nom; if copulas, goallut or jápmit is FMAINV and habitive or human Loc is found to the left. OR: if Ill or @Pron< followed by HAB are found to the left.
-
Hab>Advlcase<spred ( @<SUBJ) for Nom; it allows adverbials with Ill/Loc/Com/Ess to be found inbetween HAB and .
-
Nom>Advlcase<spred ( @<SUBJ) for Nom; it allows adverbials with Ill/Loc/Com/Ess to be found inbetween Nom and @<SUBJ.
-
<spred ( @<SUBJ) for Nom; if copulas to the left, and some kind of adverb, N Loc, time related word or Po to the left of it. OR: if Ill or @Pron< to the left, followed by copulas and the before mentioned to the left of copulas.
-
<spred ( @<SUBJ) for Nom, but not for Pers. To the left boahtit or heaŋgát as MAINV, and futher to the left is some kind of place related word, or time related word
-
<spredQst1 ( @<SUBJ) for Nom in a typically question sentence; if A) Hab, some kind of place word, Po or Nom to the left, and Qst followed by copulas to the left. B) same as a, only the Qst-pcle is attached to copulas. C) Qst to the left, with copulas to its left, but not if two Nom:s are found somewhere to the right. D) copulas to the left, and BOS to the left. E) Loc or Ill to the left, and Loc or Hab to the left of this, Qst and copulas to the left. F) Num @>N to the left, Hab, some kind of place word, Po or Nom to the left, and Qst followed by copulas to the left. NOTE) for all these rules; human, Loc or Sem/Plc not allowed to the right.
-
<spredQst2 (@<SPRED) for Nom; in a typically question sentence; differs from <spredQst1 by not beeing as restricted to the right. Though you are not allowed to be Pers or human.
-
Nom<spredQst (@<SPRED) for Nom; in a typically question sentence. Differs from <spredQst2 by letting Nom be found between SPRED and copulas
-
<spred (@<SPRED) for A Nom or N Nom if; the subject Nom is on the same side of copulas as you: on the right side of copulas
-
<spredVeara (@<SPRED) for veara + Nom; if genitive immediately to the right, and intransitive mainverb to the right of genitive
-
leftCop<spred (@<SPRED) for Nom; if copulas is the main verb to the left, and there is no Ess found to the left of cop (note that Loc is allowed between target and cop). OR: if you are Coll or Sem/Group with copulas to your left.
-
<spredLocEXPERIMENT (@<SPRED) for material Loc; if you are to the right of copulas, and the Nom to the left of copulas is not a hab-actor
-
NumTime (@<SPRED) for A Nom
-
<spredSg (@<SPRED) for Sg Nom
-
<spredPg (@<SPRED) for Pl Nom
-
<spred (@<SPRED) for Nom; if copulas to the left, and Nom or sentence boundary to the left of copulas. First one to the right is EOS.
-
<spred (@<SPRED) for N Ess
-
spredEss> (@SPRED>) for N Ess; if copulas to the right of you, and if an NP with nom-case first one to your left.
-
HABSpredSg> (@SPRED>) for Nom; if habitive first one to the left, followed by copulas.
-
GalleSpred> (@SPRED>) for Num Nom; if sentence initial
-
spredSgMII> (@SPRED>)
-
r492> (@SPRED>) for Interr Gen; consisting only of negations. You are not allowed to be MII. You are not allowed to have an adjective or noun to yor right. You are not allowed to have a verb to your right; the exception beeing an aux.
-
AdjSpredSg> (@SPRED>) for A Sg Nom; if copulas to the right, but not if A or @<SPRED are found to the right of copulas
-
SpredSg>Hab (@SPRED>) for Nom; if you are sentence initial, copulas is located to the right, and there is a habitive to the right of copulas
-
Spred>SubjInf (@SPRED>) for Nom; if copulas to the right, and the subject of copulas is an Inf to the right
-
spredCoord (@<SPRED) coordination for Nom; only if there already is a SPRED to the left of CNP. Not if there is some kind of comparison involved.
-
subj>Sgnr1 (@SUBJ>) for Nom Sg, including Indef Nom if; VFIN + Sg3 or Pl3 to the right (VFIN not allowed to the left)
- subj>Du (@SUBJ>) for dual nominatives, including Coll Nom. VFIN + Du3 to the right.
-
subj>Pl (@SUBJ>) for plural nominatives, including Coll and Sem/Group. VFIN + Pl3 to the right.
-
subj>Pl (@SUBJ>) for plural nominatives
-
subj>Sgnr2 (@SUBJ>) for Nom Sg; if VFIN + Sg3 to the right.
-
<subjSg (@<SUBJ) for Nom Sg; if VFIN Sg3 or Du2 to the left (no HAB allowed to the left).
-
f<advl (@-F<ADVL) for infinite adverbials
-
f<advl (@-F<ADVL) for infinite adverbials
-
s-boundary=advl> (@ADVL>) for ADVL that resemble s-booundaries. Mainverb to the right.
-
-fobj> (@-FOBJ>) for Acc
-
-fobj> (@-FOBJ>) for Acc
-
advl>mainV (@ADVL>) if; finite mainverb not found to the left, but the finite mainverb is found to the right.
-
<advl (@<ADVL) if; finite mainverb found to the left. Not if a comma is found immediately to the left and a finite mainverb is located somewhere to the right of this comma.
- <advlPoPr (@<ADVL) if mainverb to the left.
-
advlPoPr> (@<ADVL) if mainverb to the right.
-
advlEss> (@<ADVL) for weather and time Ess, if FMAINV to the left.
-
advl>inbetween (@ADVL>) for Adv; if inbetween two sentenceboundaries where no mainverb is present.
-
comma<advlEOS (@<ADVL) if; comma found to the left and the finite mainverb to the left of comma. To the right is the end of the sentence.
-
advlBOS> (@ADVL>) if; you are N Ill and found sentnece initially. First one to your right is a clause.
-
<advlPoEOS (@<ADVL) for Po; if you are found at the very end of a sentence. A mainverb is needed to the right though.
-
cleanupILL<advl (@<ADVL) for N Ill if; there are no boundarysymbols to your left, if you arent already @N< OR @APP-N<, and no mainverb is to yor left.
- <opredAAcc (@<OPRED) for A Acc; if an other accusative to the left, and a transtive verb to the left of it. OR: if a transitive verb to the left, and an accusative to the left of it.
sma object
- <advlEss (@<ADVL) for ESS-ADVL if; FMAINV to the left
- <spredEss (@<SPRED) for N Ess if; FMAINV to the left is intransitive or bargat
SUBJ MAPPING - leftovers
OBJ MAPPING - leftovers
HNOUN MAPPING
This (part of) documentation was generated from src/cg3/functions.cg3
src-fst-morphology-affixes-adjectives.lexc.md
Adjective inflection
Khanty
This (part of) documentation was generated from src/fst/morphology/affixes/adjectives.lexc
src-fst-morphology-affixes-adverbs.lexc.md
Adjective inflection
Khanty
This (part of) documentation was generated from src/fst/morphology/affixes/adverbs.lexc
src-fst-morphology-affixes-conjunctions.lexc.md
Conjunctions
Khanty
This (part of) documentation was generated from src/fst/morphology/affixes/conjunctions.lexc
src-fst-morphology-affixes-nouns.lexc.md
Noun inflection in Khanty
ACTUAL CASES
Start Plural
Start Dual
Possessor Indices
Single possessum
Dual possessa
Plural possessa
This (part of) documentation was generated from src/fst/morphology/affixes/nouns.lexc
src-fst-morphology-affixes-postpositions.lexc.md
Postpositions
Khanty
This (part of) documentation was generated from src/fst/morphology/affixes/postpositions.lexc
src-fst-morphology-affixes-pronouns.lexc.md
Pronoun inflection
Northern Khanty
This (part of) documentation was generated from src/fst/morphology/affixes/pronouns.lexc
src-fst-morphology-affixes-propernouns.lexc.md
Proper noun inflection
Khanty
This (part of) documentation was generated from src/fst/morphology/affixes/propernouns.lexc
src-fst-morphology-affixes-quantifiers.lexc.md
Quantifier inflection
This (part of) documentation was generated from src/fst/morphology/affixes/quantifiers.lexc
src-fst-morphology-affixes-symbols.lexc.md
Symbol affixes
This (part of) documentation was generated from src/fst/morphology/affixes/symbols.lexc
src-fst-morphology-affixes-verbs.lexc.md
Verb inflection
PRESENT
Subject
Object Singular
Object Plural
Subject
Singular Object
Dual and Plural Object
Passive
-лд-эм Ӆлд
PRETERITE
IMPERATIVE
CONDITIONAL PRESENT
REFLEXIVE
PRESENT
IMPERFECT
CONDITIONAL PRESENT
IMPERATIVE
CONDITIONAL
REFLEXIVE
INFINITIVE
-ty -infinitive added
This (part of) documentation was generated from src/fst/morphology/affixes/verbs.lexc
src-fst-morphology-phonology.twolc.md
NO CODE
! used in possessor indices
triggers
lenition
vowel raising э:ы о:у
stem-final vowel is lost in plural accusative
хасава:хасев
in combination with stem-final vowel loss тёня:тён
яля:ялэ
Surface value of linking vowel before +PxSg3
%{ЕЭ%}:е
LOSS
This (part of) documentation was generated from src/fst/morphology/phonology.twolc
src-fst-morphology-root.lexc.md
Morphology
INTRODUCTION TO MORPHOLOGICAL ANALYSER OF UNDEFINED LANGUAGE.
Analysis symbols
The morphological analyses of wordforms of NORTHERN KHANTY language are presented
in this system in terms of following symbols.
(It is highly suggested to follow existing standards when adding new tags).
These letters are hopefully are not a problem
The parts-of-speech are:
The parts of speech are further split up into:
The Usage extents are marked using following tags:
The dialect variants are expressed using the following tags:
The nominals are inflected in the following Case and Number
The possession is marked as such:
The comparative forms are:
Numerals are classified under:
Verb moods are:
Verb personal forms are:
Other verb forms are
- +Symbol = independent symbols in the text stream, like £, €, ©
Special symbols are classified with:
The verbs are syntactically split according to transitivity:
Special multiword units are analysed with:
Non-dictionary words can be recognised with:
Question and Focus particles:
Semantics are classified with
Derivations are classified under the morphophonetic form of the suffix, the
source and target part-of-speech.
Morphophonology
To represent phonologic variations in word forms we use the following
symbols in the lexicon files:
- +Symbol ! used in possessor indices
- +Symbol ! used in possessor indices
Symbols that need to be escaped on the lower side (towards twolc):
- »7: Literal »
- «7: Literal «
%[%>%] - Literal >
%[%<%] - Literal <
And following triggers to control variation
Flag diacritics
We have manually optimised the structure of our lexicon using following
flag diacritics to restrict morhpological combinatorics - only allow compounds
with verbs if the verb is further derived into a noun again:
| @P.NeedNoun.ON@ | (Dis)allow compounds with verbs unless nominalised
| @D.NeedNoun.ON@ | (Dis)allow compounds with verbs unless nominalised
| @C.NeedNoun@ | (Dis)allow compounds with verbs unless nominalised
For languages that allow compounding, the following flag diacritics are needed
to control position-based compounding restrictions for nominals. Their use is
handled automatically if combined with +CmpN/xxx tags. If not used, they will
do no harm.
| @P.CmpFrst.FALSE@ | Require that words tagged as such only appear first
| @D.CmpPref.TRUE@ | Block such words from entering ENDLEX
| @P.CmpPref.FALSE@ | Block these words from making further compounds
| @D.CmpLast.TRUE@ | Block such words from entering R
| @D.CmpNone.TRUE@ | Combines with the next tag to prohibit compounding
| @U.CmpNone.FALSE@ | Combines with the prev tag to prohibit compounding
| @P.CmpOnly.TRUE@ | Sets a flag to indicate that the word has passed R
| @D.CmpOnly.FALSE@ | Disallow words coming directly from root.
Use the following flag diacritics to control downcasing of derived proper
nouns (e.g. Finnish Pariisi -> pariisilainen). See e.g. North Sámi for how to use
these flags. There exists a ready-made regex that will do the actual down-casing
given the proper use of these flags.
| @U.Cap.Obl@ | Allowing downcasing of derived names: deatnulasj.
| @U.Cap.Opt@ | Allowing downcasing of derived names: deatnulasj.
The word forms in Khanty language start from the lexeme roots of basic
word classes, or optionally from prefixes:
This (part of) documentation was generated from src/fst/morphology/root.lexc
src-fst-morphology-stems-exceptions.lexc.md
Adjectives
Adverb
Conjunctions
Nouns
Postpositions
Pronouns
Quantifiers
Verbs
This (part of) documentation was generated from src/fst/morphology/stems/exceptions.lexc
src-fst-phonetics-txt2ipa.xfscript.md
retroflex plosive, voiceless t ʈ 0288, 648 (
= ASCII 096)
retroflex plosive, voiced d ɖ 0256, 598
labiodental nasal F ɱ 0271, 625
retroflex nasal n
ɳ 0273, 627
palatal nasal J ɲ 0272, 626
velar nasal N ŋ 014B, 331
uvular nasal N\ ɴ 0274, 628
bilabial trill B\ ʙ 0299, 665
uvular trill R\ ʀ 0280, 640
alveolar tap 4 ɾ 027E, 638
retroflex flap r ɽ 027D, 637
bilabial fricative, voiceless p\ ɸ 0278, 632
bilabial fricative, voiced B β 03B2, 946
dental fricative, voiceless T θ 03B8, 952
dental fricative, voiced D ð 00F0, 240
postalveolar fricative, voiceless S ʃ 0283, 643
postalveolar fricative, voiced Z ʒ 0292, 658
retroflex fricative, voiceless s
ʂ 0282, 642
retroflex fricative, voiced z` ʐ 0290, 656
palatal fricative, voiceless C ç 00E7, 231
palatal fricative, voiced j\ ʝ 029D, 669
velar fricative, voiced G ɣ 0263, 611
uvular fricative, voiceless X χ 03C7, 967
uvular fricative, voiced R ʁ 0281, 641
pharyngeal fricative, voiceless X\ ħ 0127, 295
pharyngeal fricative, voiced ?\ ʕ 0295, 661
glottal fricative, voiced h\ ɦ 0266, 614
alveolar lateral fricative, vl. K
alveolar lateral fricative, vd. K\
labiodental approximant P (or v)
alveolar approximant r\
retroflex approximant r`
velar approximant M\
retroflex lateral approximant l`
palatal lateral approximant L
velar lateral approximant L
Clicks
bilabial O\ (O = capital letter)
dental |
(post)alveolar !\
palatoalveolar =\
alveolar lateral ||
Ejectives, implosives
ejective > e.g. ejective p p>
implosive < e.g. implosive b b<
Vowels
close back unrounded M
close central unrounded 1
close central rounded }
lax i I
lax y Y
lax u U
close-mid front rounded 2
close-mid central unrounded @\
close-mid central rounded 8
close-mid back unrounded 7
schwa ə @
open-mid front unrounded E
open-mid front rounded 9
open-mid central unrounded 3
open-mid central rounded 3\
open-mid back unrounded V
open-mid back rounded O
ash (ae digraph) {
open schwa (turned a) 6
open front rounded &
open back unrounded A
open back rounded Q
Other symbols
voiceless labial-velar fricative W
voiced labial-palatal approx. H
voiceless epiglottal fricative H\
voiced epiglottal fricative <\
epiglottal plosive >\
alveolo-palatal fricative, vl. s\
alveolo-palatal fricative, voiced z\
alveolar lateral flap l\
simultaneous S and x x\
tie bar _
Suprasegmentals
primary stress “
secondary stress %
long :
half-long :\
extra-short _X
linking mark -
Tones and word accents
level extra high _T
level high _H
level mid _M
level low _L
level extra low _B
downstep !
upstep ^ (caret, circumflex)
contour, rising
contour, falling _F
contour, high rising _H_T
contour, low rising _B_L
contour, rising-falling _R_F
(NB Instead of being written as diacritics with _, all prosodic
marks can alternatively be placed in a separate tier, set off
by < >, as recommended for the next two symbols.)
global rise
global fall
Diacritics
voiceless 0 (0 = figure), e.g. n_0
voiced _v
aspirated _h
more rounded _O (O = letter)
less rounded _c
advanced _+
retracted _-
centralized _”
syllabic = (or _=) e.g. n= (or n=)
non-syllabic _^
rhoticity `
breathy voiced _t
creaky voiced _k
linguolabial _N
labialized _w
palatalized ‘ (or _j) e.g. t’ (or t_j)
velarized _G
pharyngealized _?\
dental d
apical _a
laminal _m
nasalized ~ (or _~) e.g. A~ (or A~)
nasal release _n
lateral release _l
no audible release _}
velarized or pharyngealized _e
velarized l, alternatively 5
raised _r
lowered _o
advanced tongue root _A
retracted tongue root _q
This (part of) documentation was generated from src/fst/phonetics/txt2ipa.xfscript
src-fst-root-from-old-infra.lexc.md
Derivations
This (part of) documentation was generated from src/fst/root-from-old-infra.lexc
src-fst-transcriptions-transcriptor-abbrevs2text.lexc.md
We describe here how abbreviations are in Khanty are read out, e.g.
for text-to-speech systems.
For example:
- s.:syntynyt # ;
- os.:omaa% sukua # ;
- v.:vuosi # ;
- v.:vuonna # ;
- esim.:esimerkki # ;
- esim.:esimerkiksi # ;
This (part of) documentation was generated from src/fst/transcriptions/transcriptor-abbrevs2text.lexc
src-fst-transcriptions-transcriptor-numbers-digit2text.lexc.md
File containing Nenets numerals to cyphers
- HUNDREDSM ; = 200M
- 1:юрˮ HUNDREDM ; = 100M
- TENSM ; = 20-99M
- TEENSM ; = 10-19M
- 1MILJON ; = 1M
- ONESM ; = 1-9M
- 1GIGA ; = 1G
- ONESG ; = 1-9G
- HUNDREDST ; = 200000-999999
- 1:юрˮ HUNDREDT ; = 100000-100999
- TENST ; = 20000-99999,10000-10999
- TEENST ; = 11000-19999
- ONEST ; = 2000-9999
- 1:ёнарˮ% THOUSAND ; = 1000-1999
- ORD_THOUSAND ; =
- ORD_HUNDREDS ; =
- UNDERTHOUSAND ; = 100-999
- TENS ; = 20-99
- TEENS ; = 10-19
- ONES ; = 1-9
This (part of) documentation was generated from src/fst/transcriptions/transcriptor-numbers-digit2text.lexc
[ L A N G U A G E ] G R A M M A R C H E C K E R
DELIMITERS
This section lists all the tags inherited from the fst, and used as tags
in the syntactic analysis. The next section, Sets, contains sets defined
on the basis of the tags listed here, those set names are not visible in the output.
Beginning and end of sentence
BOS
EOS
N
A
Adv
V
Pron
CS
CC
CC-CS
Po
Pr
Pcle
Num
Interj
ABBR
ACR
CLB
LEFT
RIGHT
WEB
PPUNCT
PUNCT
COMMA
¶
Pers
Dem
Interr
Indef
Recipr
Refl
Rel
Coll
NomAg
Prop
Allegro
Arab
Romertall
Nom
Acc
Gen
Ill
Loc
Com
Ess
Ess
Sg
Du
Pl
Cmp/SplitR
Cmp/SgNom Cmp/SgGen
Cmp/SgGen
PxSg1
PxSg2
PxSg3
PxDu1
PxDu2
PxDu3
PxPl1
PxPl2
PxPl3
Px
Comp
Superl
Attr
Ord
Qst
IV
TV
Prt
Prs
Ind
Pot
Cond
Imprt
ImprtII
Sg1
Sg2
Sg3
Du1
Du2
Du3
Pl1
Pl2
Pl3
Inf
ConNeg
Neg
PrfPrc
VGen
PrsPrc
Ger
Sup
Actio
VAbess
Err/Orth
Sem/Act
Sem/Ani
Sem/Atr
Sem/Body
Sem/Clth
Sem/Domain
Sem/Feat-phys
Sem/Fem
Sem/Group
Sem/Lang
Sem/Mal
Sem/Measr
Sem/Money
Sem/Obj
Sem/Obj-el
Sem/Org
Sem/Perc-emo
Sem/Plc
Sem/Sign
Sem/State-sick
Sem/Sur
Sem/Time
Sem/Txt
HUMAN
PROP-ATTR
PROP-SUR
TIME-N-SET
@+FAUXV
@+FMAINV
@-FAUXV
@-FMAINV
@-FSUBJ>
@-F<OBJ
@-FOBJ>
@-FSPRED<OBJ
@-F<ADVL
@-FADVL>
@-F<SPRED
@-F<OPRED
@-FSPRED>
@-FOPRED>
@>ADVL
@ADVL<
@<ADVL
@ADVL>
@ADVL
@HAB>
@<HAB
@>N
@Interj
@N<
@>A
@P<
@>P
@HNOUN
@INTERJ
@>Num
@Pron<
@>Pron
@Num<
@OBJ
@<OBJ
@OBJ>
@OPRED
@<OPRED
@OPRED>
@PCLE
@COMP-CS<
@SPRED
@<SPRED
@SPRED>
@SUBJ
@<SUBJ
@SUBJ>
SUBJ
SPRED
OPRED
@PPRED
@APP
@APP-N<
@APP-Pron<
@APP>Pron
@APP-Num<
@APP-ADVL<
@VOC
@CVP
@CNP
OBJ
-OTHERS
SYN-V
@X
## Sets containing sets of lists and tags
This part of the file lists a large number of sets based partly upon the tags defined above, and
partly upon lexemes drawn from the lexicon.
See the sourcefile itself to inspect the sets, what follows here is an overview of the set types.
### Sets for Single-word sets
INITIAL
### Sets for word or not
WORD
NOT-COMMA
### Case sets
ADLVCASE
CASE-AGREEMENT
CASE
NOT-NOM
NOT-GEN
NOT-ACC
### Verb sets
NOT-V
### Sets for finiteness and mood
REAL-NEG
MOOD-V
NOT-PRFPRC
### Sets for person
SG1-V
SG2-V
SG3-V
DU1-V
DU2-V
DU3-V
PL1-V
PL2-V
PL3-V
### Pronoun sets
### Adjectival sets and their complements
### Adverbial sets and their complements
### Sets of elements with common syntactic behaviour
### NP sets defined according to their morphosyntactic features
### The PRE-NP-HEAD family of sets
These sets model noun phrases (NPs). The idea is to first define whatever can
occur in front of the head of the NP, and thereafter negate that with the
expression **WORD - premodifiers**.
### Border sets and their complements
### Grammarchecker sets
* * *
This (part of) documentation was generated from [tools/grammarcheckers/grammarchecker.cg3](https://github.com/giellalt/lang-kca/blob/main/tools/grammarcheckers/grammarchecker.cg3)
---
# tools-tokenisers-tokeniser-disamb-gt-desc.pmscript.md
# Tokeniser for kca
Usage:
```
$ make
$ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
$ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
$ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
$ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
```
Pmatch documentation:
<https://github.com/hfst/hfst/wiki/HfstPmatch>
Characters which have analyses in the lexicon, but can appear without spaces
before/after, that is, with no context conditions, and adjacent to words:
* Punct contains ASCII punctuation marks
* The symbol after m-dash is soft-hyphen `U+00AD`
* The symbol following {•} is byte-order-mark / zero-width no-break space
`U+FEFF`.
Whitespace contains ASCII white space and
the List contains some unicode white space characters
* En Quad U+2000 to Zero-Width Joiner U+200d'
* Narrow No-Break Space U+202F
* Medium Mathematical Space U+205F
* Word joiner U+2060
Apart from what's in our morphology, there are
1. unknown word-like forms, and
2. unmatched strings
We want to give 1) a match, but let 2) be treated specially by
`hfst-tokenise -a`
Unknowns are made of:
* lower-case ASCII
* upper-case ASCII
* select extended latin symbols
ASCII digits
* select symbols
Cyrillic letters and combining diacritics, taken from
https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode, the
* Combining diacritics as individual symbols,
* various symbols from Private area (probably Microsoft),
so far:
* U+F0B7 for "x in box"
## Unknown handling
Unknowns are tagged ?? and treated specially with `hfst-tokenise`
hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and
remove empty analyses from other readings. Empty readings are also
legal in CG, they get a default baseform equal to the wordform, but
no tag to check, so it's safer to let hfst-tokenise handle them.
Finally we mark as a token any sequence making up a:
* known word in context
* unknown (OOV) token in context
* sequence of word and punctuation
* URL in context
* * *
This (part of) documentation was generated from [tools/tokenisers/tokeniser-disamb-gt-desc.pmscript](https://github.com/giellalt/lang-kca/blob/main/tools/tokenisers/tokeniser-disamb-gt-desc.pmscript)
---
# tools-tokenisers-tokeniser-gramcheck-gt-desc.pmscript.md
# Grammar checker tokenisation for kca
Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc)
Then just:
```
$ make
$ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
```
More usage examples:
```
$ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
$ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
$ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
```
Pmatch documentation:
<https://github.com/hfst/hfst/wiki/HfstPmatch>
Characters which have analyses in the lexicon, but can appear without spaces
before/after, that is, with no context conditions, and adjacent to words:
* Punct contains ASCII punctuation marks
* The symbol after m-dash is soft-hyphen `U+00AD`
* The symbol following {•} is byte-order-mark / zero-width no-break space
`U+FEFF`.
Whitespace contains ASCII white space and
the List contains some unicode white space characters
* En Quad U+2000 to Zero-Width Joiner U+200d'
* Narrow No-Break Space U+202F
* Medium Mathematical Space U+205F
* Word joiner U+2060
Apart from what's in our morphology, there are
1) unknown word-like forms, and
2) unmatched strings
We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a
* select extended latin symbols
* select symbols
* various symbols from Private area (probably Microsoft),
so far:
* U+F0B7 for "x in box"
TODO: Could use something like this, but built-in's don't include šžđčŋ:
Simply give an empty reading when something is unknown:
hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and
remove empty analyses from other readings. Empty readings are also
legal in CG, they get a default baseform equal to the wordform, but
no tag to check, so it's safer to let hfst-tokenise handle them.
Finally we mark as a token any sequence making up a:
* known word in context
* unknown (OOV) token in context
* sequence of word and punctuation
* URL in context
* * *
This (part of) documentation was generated from [tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript](https://github.com/giellalt/lang-kca/blob/main/tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript)
---
# tools-tokenisers-tokeniser-tts-cggt-desc.pmscript.md
# TTS tokenisation for smj
Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc)
Then just:
```sh
make
echo "ja, ja" \
| hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
```
More usage examples:
```sh
echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa \
boasttu olmmoš, man mielde lahtuid." \
| hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" \
| hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
echo "márffibiillagáffe" \
| hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst
```
Pmatch documentation:
<https://kitwiki.csc.fi/twiki/bin/view/KitWiki/HfstPmatch>
Characters which have analyses in the lexicon, but can appear without spaces
before/after, that is, with no context conditions, and adjacent to words:
* Punct contains ASCII punctuation marks
* The symbol after m-dash is soft-hyphen `U+00AD`
* The symbol following {•} is byte-order-mark / zero-width no-break space
`U+FEFF`.
Whitespace contains ASCII white space and
the List contains some unicode white space characters
* En Quad U+2000 to Zero-Width Joiner U+200d'
* Narrow No-Break Space U+202F
* Medium Mathematical Space U+205F
* Word joiner U+2060
Apart from what's in our morphology, there are
1) unknown word-like forms, and
2) unmatched strings
We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a
* select extended latin symbols
* select symbols
* various symbols from Private area (probably Microsoft),
so far:
* U+F0B7 for "x in box"
TODO: Could use something like this, but built-in's don't include šžđčŋ:
Simply give an empty reading when something is unknown:
hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and
remove empty analyses from other readings. Empty readings are also
legal in CG, they get a default baseform equal to the wordform, but
no tag to check, so it's safer to let hfst-tokenise handle them.
Needs hfst-tokenise to output things differently depending on the tag they get
* * *
This (part of) documentation was generated from [tools/tokenisers/tokeniser-tts-cggt-desc.pmscript](https://github.com/giellalt/lang-kca/blob/main/tools/tokenisers/tokeniser-tts-cggt-desc.pmscript)