Nenets NLP Grammar

Finite state and Constraint Grammar based analysers, proofing tools and other resources

View the project on GitHub giellalt/lang-yrk

Page Content

  • src-fst-morphology-affixes-quantifiers.lexc.md
  • Quantifier inflection
  • src-fst-morphology-affixes-symbols.lexc.md
  • Symbol affixes
  • src-fst-morphology-affixes-verbs.lexc.md
  • Nenets Verb inflection
  • src-fst-morphology-phonology.twolc.md
  • Nenets twol file
  • Rules
  • src-fst-morphology-root.lexc.md
  • Morphology
  • Definitions for Multichar_Symbols
  • The Root lexicon
  • src-fst-morphology-stems-verbs_newwords.lexc.md
  • src-fst-phonetics-txt2ipa.xfscript.md
  • src-fst-transcriptions-transcriptor-abbrevs2text.lexc.md
  • src-fst-transcriptions-transcriptor-numbers-digit2text.lexc.md
  • File containing Nenets numerals to cyphers
  • tools-grammarcheckers-grammarchecker.cg3.md
  • DELIMITERS
  • TAGS AND SETS
  • Nenets language model documentation

    All doc-comment documentation in one large file.


    src-cg3-functions.cg3.md

    These sets model noun phrases (NPs). The idea is to first define whatever can occur in front of the head of the NP, and thereafter negate that with the expression WORD - premodifiers.

    The set NOT-NPMOD is used to find barriers between NPs. Typical usage: … (*1 N BARRIER NPT-NPMOD) … meaning: Scan to the first noun, ignoring anything that can be part of the noun phrase of that noun (i.e., “scan to the next NP head”)

    These were the set types.

    HABITIVE MAPPING

    sma object

    SUBJ MAPPING - leftovers

    OBJ MAPPING - leftovers

    HNOUN MAPPING


    This (part of) documentation was generated from src/cg3/functions.cg3


    src-fst-morphology-affixes-adjectives.lexc.md

    Adjective inflection Nenets adjectives.

    **LEXICON æLEXNAME@ to #

    Check this

    LEXICON A_САБЦЬ тир: 12 !!ProsSg -увна What makes this different from N_ТИРЕ?


    This (part of) documentation was generated from src/fst/morphology/affixes/adjectives.lexc


    src-fst-morphology-affixes-adpositions.lexc.md

    Adposition inflection

    Nenets adpositions inflect in person (and some in local cases).

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@


    This (part of) documentation was generated from src/fst/morphology/affixes/adpositions.lexc


    src-fst-morphology-affixes-adverbs.lexc.md

    Adverbs

    Nenets adverbs…


    This (part of) documentation was generated from src/fst/morphology/affixes/adverbs.lexc


    src-fst-morphology-affixes-clitics.lexc.md

    Clitics inflection

    Nenets clitics…

    **LEXICON æLEXNAME@ optional +Qst

    **LEXICON æLEXNAME@ leads to #.


    This (part of) documentation was generated from src/fst/morphology/affixes/clitics.lexc


    src-fst-morphology-affixes-descriptives.lexc.md

    Descriptives

    Nenets descriptives…

    LEXICON æLEXNAME@ adds the tag **+Descr


    This (part of) documentation was generated from src/fst/morphology/affixes/descriptives.lexc


    src-fst-morphology-affixes-interjections.lexc.md

    Interjections

    Nenets interjections…

    **LEXICON æLEXNAME@ just goes to #


    This (part of) documentation was generated from src/fst/morphology/affixes/interjections.lexc


    src-fst-morphology-affixes-nouns.lexc.md

    Noun inflection

    Nenets nouns inflect in cases.

    **LEXICON æLEXNAME@

    та:та

    No n2d

    Where is the variation?

    Where is the variation?

    Where is the variation?

    NMN

    9

    NOMINALS “NMN”

    THREE-SYLLABLE VOWEL-FINAL STEMS

    таˮ:та

    варˮ:вар Yaml: N-tarTS | —

    The singular accusative stem

    Start Plural

    What is assumed on the basis of the plural accusative

    Possessor Indices

    SINUGLAR CASES

    The next line in +Sg+Pros+PxDu2:%>%{увм%}А2нандиʼ must be removed

    Dual possessa

    Dual possessa

    Plural Possessa

    Conjugation of nouns and adjectives

    NEW

    NEW-SG-LOC_Кна/Кана


    This (part of) documentation was generated from src/fst/morphology/affixes/nouns.lexc


    src-fst-morphology-affixes-pronouns.lexc.md

    Pronoun inflection

    Nenets pronouns inflection

    **LEXICON æLEXNAME@ for the unclassified ones

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@ etc.

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@ etc.

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@ etc.

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@ +Interr+Sem/Hum: POSSESSA-PLURAL ; +Pl+Dat, +Pl+Loc, +Pl+Abl

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@


    This (part of) documentation was generated from src/fst/morphology/affixes/pronouns.lexc


    src-fst-morphology-affixes-propernouns.lexc.md

    Proper noun inflection

    Nenets proper nouns inflect in the same cases as regular nouns, but with a XXX as separator.

    **LEXICON æLEXNAME@ for bunclassified ones

    ONE-SYLLABLE VOWEL-FINAL STEM

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    ONE-SYLLABLE CONSONANT-FINAL STEM

    **LEXICON æLEXNAME@

    TWO-SYLLABLE CONSONANT-FINAL STEM

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    THREE-SYLLABLE VOWEL-FINAL STEM

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@ Here we need some kind of vowel harmony


    This (part of) documentation was generated from src/fst/morphology/affixes/propernouns.lexc


    src-fst-morphology-affixes-quantifiers.lexc.md

    Quantifier inflection

    Nenets quantifiers …

    LEXICON NUM_ЕД ед: 13

    we need to get away from these: NUM_VOW and NUM_CONS it’s done


    This (part of) documentation was generated from src/fst/morphology/affixes/quantifiers.lexc


    src-fst-morphology-affixes-symbols.lexc.md

    Symbol affixes

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@

    **LEXICON æLEXNAME@


    This (part of) documentation was generated from src/fst/morphology/affixes/symbols.lexc


    src-fst-morphology-affixes-verbs.lexc.md

    Nenets Verb inflection

    LEXICON V_ for unassigned verbs

    ONE-SYLLABLE STEMS WITH STEM-FINAL VOWEL

    LEXICON IV_МЭ

    LEXICON TV_МЭ

    LEXICON VR_МЭ

    LEXICON IV_МЫ

    LEXICON TV_МЫ

    LEXICON VA_НЁ

    LEXICON VA_НИ

    LEXICON IV_НИ

    LEXICON IV_НО

    LEXICON IV_НУ

    LEXICON IV_НУ-Pal/Var

    LEXICON VR_НУ

    LEXICON VR_НУ-Pal/Var

    LEXICON TV_НУ

    LEXICON TV_НУ-Pal/Var

    LEXICON IV_НУ/ТУ

    LEXICON IV_ҢЭ-Pal/Var

    LEXICON IV_ТУ

    LEXICON TV_ТУ

    LEXICON IV_ХО

    LEXICON TV_ХО

    LEXICON VR_ХО-Pal/Var

    LEXICON IV_ХЭ

    TWO-SYLLABLE STEMS WITH STEM-FINAL VOWEL

    LEXICON TV_НАМДА

    LEXICON VR_НАМДА

    LEXICON IV_ВАДЮ

    LEXICON TV_ВАДЮ

    LEXICON IV_ИЛЕ

    LEXICON IV_ИЛЕ-Pal/Var Work needed for variation 2013-03-04

    LEXICON VR_ИЛЕ

    LEXICON VR_ИЛЕ-Pal/Var Work needed for variation 2013-09-05

    LEXICON TV_ИЛЕ

    LEXICON IV_НЕНЫ

    LEXICON IV_ҢЭСУ

    LEXICON TV_ҢЭСУ

    LEXICON IV_ПЭБЮ

    LEXICON IV_ПЭБЮ-Pal/Var

    LEXICON TV_ПЭБЮ

    LEXICON IV_ХАДАБАСЬ

    LEXICON TV_ХАДАБАСЬ

    LEXICON VR_ХАДА

    LEXICON IV_ЮХУ

    LEXICON TV_ЮХУ

    THREE-SYLLABLE STEMS WITH STEM-FINAL VOWEL

    LEXICON IV_ЛАХАНА

    LEXICON IV_ЛАХАНА-Pal/Var

    LEXICON VR_ЛАХАНА

    LEXICON TV_ЛАХАНА

    LEXICON TV_ЛАХАНА-Pal/Var

    LEXICON IV_ҢЭДАРА LEXICON TV_ҢЭДАРА LEXICON VR_ҢЭДАРА

    LEXICON IV_СЯНАКУ

    LEXICON IV_ЯˮАВЛУ

    LEXICON IV_ЯˮАВЛУ-Pal/Var

    LEXICON TV_ЯˮАВЛУ

    ONE-SYLLABLE STEMS WITH STEM-FINAL CONSONANT

    LEXICON IV_МИН

    LEXICON IV_МИН-Pal/Var

    LEXICON TV_МИН

    LEXICON VR_МИН

    LEXICON IV_САС

    TWO-SYLLABLE STEMS WITH STEM-FINAL CONSONANT

    AFTER +TV, +IV, +Aux, +Refl

    Not yet written.

    CONJUGATION BY STEM TYPE

    ONE-SYLLABLE STEMS WITH STEM-FINAL VOWEL

    V_refl

    V_refl 2013-03-04

    TWO-SYLLABLE STEMS WITH STEM-FINAL VOWEL

    V_refl 2013-03-04

    V_refl

    THREE-SYLLABLE STEMS WITH STEM-FINAL VOWEL

    а > э in Pl Oc

    V_refl 2013-03-04 а > ы (ъя before х)

    ONE-SYLLABLE STEMS WITH STEM-FINAL CONSONANT

    V_refl 2013-03-04

    TWO-SYLLABLE STEMS WITH STEM-FINAL CONSONANT

    V_refl 2013-03-04

    Mutual verbal conjugation

    Preterite 1 # This should be for ConjPrt and NarrPrt also.

    Other moods сь in 2nd, 4th etc non-final syllable

    Mutual person ending for nonindicative moods

    This need separating 2013-04-23

    Object

    Reflexive

    Reflexive Preterite

    Non-finites

    no vowel loss in stem

    ◊_subj

    ◊_obj

    ◊_subj

    ◊_obj

    ◊_obj


    This (part of) documentation was generated from src/fst/morphology/affixes/verbs.lexc


    src-fst-morphology-phonology.twolc.md

    Nenets twol file

    This file documents the phonology.twolc file

    Letters of the alphabet

    Archiphonemes for vowels

    Archiphonemes for glottals

    triggers

    Boundary symbols

    Rules

    к: -> г after nasal glottal

    к: to х after Vowels

    nTod

    %{нңʼØ%}:н GlottalNasalToSurface

    %{нңʼØ%}:ң GlottalNasalToSurface

    %{йнңъʼØ%}:ң GlottalNasalToSurface

    %{йнңъʼØ%}:н GlottalNasalToSurface

    %{йнңъʼØ%}:ъ GlottalNasalToSurface

    %{нңʼØ%}:0 GlottalNasalToSurface

    %{йнңъʼØ%}:0 GlottalNasalToSurface

    %{дˮØ%}:0 GlottalNonNasalToZERO LEFT

    RIGHT ARROW ONLY %{дˮØ%}:0

    С1:0 GlottalNonNasalToZERO

    С1:ˮ UnderlyingToGlottal

    %{дˮØ%}:ˮ UnderlyingToGlottal LEFT

    RIGHT ARROW ONLY %{дˮØ%}:ˮ

    %{нңʼØ%}:ʼ UnderlyingToGlottal

    с => ц affrication after Stop

    с => з after nasal

    **v To m **

    %{рл%}:л

    %{рл%}:р

    %{рл%}:р

    0:а in single-syllable

    ʼ:н

    Second Vowel Center °:у in Pros LEFT

    ә:у in extra short to у

    ә:о in extra short to о

    ә:и in extra short to и

    ә:ы in extra short to ы

    ә:э in extra short to э

    ә:а in extra short to а

    0:а in Pros

    0:э in Pros

    0:о between two glottals

    0:и after glottal

    0:ы between two glottals

    0:у between two glottals

    0:у between two glottals

    а:я VowelPalatalization left

    %{ая%}:я VowelPalatalization left

    %{ая%}:я VowelPalatalization right

    %{оё%}:ё VowelPalatalization right

    The rule %{ауоэØ%} <-> у:

    The rule %{ауоэØ%} -> о:

    The rule %{ауоэØ%} -> а:

    %{ауоэØ%} -> а

    %{ауоэØ%} -> а

    %{ауоэØ%} -> а

    VOWEL LOWERING

    i2e StemFinalYToE LEFT

    а:э in stem-internal position

    a:o in я

    a:i in мараңгы

    a:i in ңуда

    о:у word-final position

    я:и word-final position

    я:е in яха

    а:у яˮавла:яˮавлу LEFT

    а:у яˮавла:яˮавлу RIGHT

    а:ю яˮавла:яˮавлу

    а:ю яˮавла:яˮавлу

    а:ю яˮавла:яˮавлу

    я:0 Stem vowel loss in plural accusative

    CONSONANT CHANGES

    mToV

    **m loss before Labial followed by nonglottal cons or vows **

    **н loss after glottal before д: **

    н loss after glottal before д: ** **r To l after nasal glottal

    r To 0 before glottal and р or л

    n to t after non-nasal glottal

    d to t after non-nasal glottal

    b to p after non-nasal glottal

    ң:0 Before в:м +Sg+Acc+PxPl1:%>ваˮ

    The rule ъ:0 with Conj after vowels:

    In the first context …

    In the second context …

    ˮ:0

    SURFACE CONSONANT

    х:с in яха

    С1:с GlottalNonNasalToSurface

    %{дˮØ%}:д GlottalNonNasalToSurface

    %{дˮØ%}:д GlottalNonNasalToSurface

    ь:0 with GlottalNonNasalToZERO


    This (part of) documentation was generated from src/fst/morphology/phonology.twolc


    src-fst-morphology-root.lexc.md

    Morphology

    INTRODUCTION TO THE MORPHOLOGICAL ANALYSER OF NENETS

    Definitions for Multichar_Symbols

    Analysis symbols

    The morphological analyses of wordforms of the TUNDRA NENETS language are presented in this system in terms of following the symbols. (It is highly suggested to follow existing standards when adding new tags).

    The parts of speech are further split up into:

    Adv

    The Usage extents are marked using the following tags:

    Dialects

    The nominals are inflected in the following Case and Number

    are these needed?:

    derivative suffixes before case endings

    The possession is marked as such:

    The comparative forms are:

    Numerals are classified under:

    Verb moods are:

    Verb tenses are:

    Verb personal forms are:

    Subject

    One of the two following

    Object

    Other verb forms are

    Abbreviated words are classified with:

    Symbols that need to be escaped on the lower side (towards twolc):

    Special symbols are classified with:

    The verbs are syntactically split according to transitivity:

    Special multiword units are analysed with:

    Non-dictionary words can be recognised with:

    Question and Focus particles:

    Semantics are classified with

    Derivations are classified under the morphophonetic form of the suffix, the source and target part-of-speech.

    Morphophonology

    To represent phonologic variations in word forms we use the following symbols in the lexicon files:

    And the following triggers to control variation:

    Protoletters for xfst:

    This is the schwa or reduced vowel occurring after x in case endings

    These are proto-glottals

    Triggers

    We have manually optimised the structure of our lexicon using the following flag diacritics to restrict morhpological combinatorics:

    Object conjugation

    The Root lexicon

    The word forms in Nenets start from the lexeme roots of basic word classes, or optionally from prefixes:


    This (part of) documentation was generated from src/fst/morphology/root.lexc


    src-fst-morphology-stems-verbs_newwords.lexc.md

    This is where new words are added as lexc entries before they are added to the xml source files. V_ “FinnishTRANSLATION” ;

    CONTINUE BELOW


    This (part of) documentation was generated from src/fst/morphology/stems/verbs_newwords.lexc


    src-fst-phonetics-txt2ipa.xfscript.md

    retroflex plosive, voiceless t ʈ 0288, 648 ( = ASCII 096) retroflex plosive, voiced d ɖ 0256, 598 labiodental nasal F ɱ 0271, 625 retroflex nasal n ɳ 0273, 627 palatal nasal J ɲ 0272, 626 velar nasal N ŋ 014B, 331 uvular nasal N\ ɴ 0274, 628

    bilabial trill B\ ʙ 0299, 665 uvular trill R\ ʀ 0280, 640 alveolar tap 4 ɾ 027E, 638 retroflex flap r ɽ 027D, 637 bilabial fricative, voiceless p\ ɸ 0278, 632 bilabial fricative, voiced B β 03B2, 946 dental fricative, voiceless T θ 03B8, 952 dental fricative, voiced D ð 00F0, 240 postalveolar fricative, voiceless S ʃ 0283, 643 postalveolar fricative, voiced Z ʒ 0292, 658 retroflex fricative, voiceless s ʂ 0282, 642 retroflex fricative, voiced z` ʐ 0290, 656 palatal fricative, voiceless C ç 00E7, 231 palatal fricative, voiced j\ ʝ 029D, 669 velar fricative, voiced G ɣ 0263, 611 uvular fricative, voiceless X χ 03C7, 967 uvular fricative, voiced R ʁ 0281, 641 pharyngeal fricative, voiceless X\ ħ 0127, 295 pharyngeal fricative, voiced ?\ ʕ 0295, 661 glottal fricative, voiced h\ ɦ 0266, 614

    alveolar lateral fricative, vl. K alveolar lateral fricative, vd. K\

    labiodental approximant P (or v) alveolar approximant r\ retroflex approximant r` velar approximant M\

    retroflex lateral approximant l` palatal lateral approximant L velar lateral approximant L
    Clicks

    bilabial O\ (O = capital letter) dental |
    (post)alveolar !\ palatoalveolar =\ alveolar lateral ||
    Ejectives, implosives

    ejective > e.g. ejective p p> implosive < e.g. implosive b b< Vowels

    close back unrounded M close central unrounded 1 close central rounded } lax i I lax y Y lax u U

    close-mid front rounded 2 close-mid central unrounded @\ close-mid central rounded 8 close-mid back unrounded 7

    schwa ə @

    open-mid front unrounded E open-mid front rounded 9 open-mid central unrounded 3 open-mid central rounded 3\ open-mid back unrounded V open-mid back rounded O

    ash (ae digraph) { open schwa (turned a) 6

    open front rounded & open back unrounded A open back rounded Q Other symbols

    voiceless labial-velar fricative W voiced labial-palatal approx. H voiceless epiglottal fricative H\ voiced epiglottal fricative <\ epiglottal plosive >\

    alveolo-palatal fricative, vl. s\ alveolo-palatal fricative, voiced z\ alveolar lateral flap l\ simultaneous S and x x\ tie bar _ Suprasegmentals

    primary stress “ secondary stress % long : half-long :\ extra-short _X linking mark -
    Tones and word accents

    level extra high _T level high _H level mid _M level low _L level extra low _B downstep ! upstep ^ (caret, circumflex)

    contour, rising contour, falling _F contour, high rising _H_T contour, low rising _B_L

    contour, rising-falling _R_F (NB Instead of being written as diacritics with _, all prosodic marks can alternatively be placed in a separate tier, set off by < >, as recommended for the next two symbols.) global rise global fall Diacritics

    voiceless 0 (0 = figure), e.g. n_0 voiced _v aspirated _h more rounded _O (O = letter) less rounded _c advanced _+ retracted _- centralized _” syllabic = (or _=) e.g. n= (or n=) non-syllabic _^ rhoticity `

    breathy voiced _t creaky voiced _k linguolabial _N labialized _w palatalized ‘ (or _j) e.g. t’ (or t_j) velarized _G pharyngealized _?\

    dental d apical _a laminal _m nasalized ~ (or _~) e.g. A~ (or A~) nasal release _n lateral release _l no audible release _}

    velarized or pharyngealized _e velarized l, alternatively 5 raised _r lowered _o advanced tongue root _A retracted tongue root _q


    This (part of) documentation was generated from src/fst/phonetics/txt2ipa.xfscript


    src-fst-transcriptions-transcriptor-abbrevs2text.lexc.md

    We describe here how abbreviations are in Nenets are read out, e.g. for text-to-speech systems.

    For example:


    This (part of) documentation was generated from src/fst/transcriptions/transcriptor-abbrevs2text.lexc


    src-fst-transcriptions-transcriptor-numbers-digit2text.lexc.md

    File containing Nenets numerals to cyphers


    This (part of) documentation was generated from src/fst/transcriptions/transcriptor-numbers-digit2text.lexc


    tools-grammarcheckers-grammarchecker.cg3.md

    T U N D R A N E N E T S G R A M M A R C H E C K E R

    DELIMITERS

    TAGS AND SETS

    Tags

    This section lists all the tags inherited from the fst, and used as tags in the syntactic analysis. The next section, Sets, contains sets defined on the basis of the tags listed here, those set names are not visible in the output.

    Beginning and end of sentence

    BOS EOS

    Parts of speech tags

    N A Adv V Pron CS CC CC-CS Po Pr Pcle Num Interj ABBR ACR CLB LEFT RIGHT WEB PPUNCT PUNCT

    COMMA ¶

    Tags for POS sub-categories

    Pers Dem Interr Indef Recipr Refl Rel Coll NomAg Prop Allegro Arab Romertall

    Tags for morphosyntactic properties

    Nom Acc Gen Ill Loc Com Ess Ess Sg Du Pl Cmp/SplitR Cmp/SgNom Cmp/SgGen Cmp/SgGen PxSg1 PxSg2 PxSg3 PxDu1 PxDu2 PxDu3 PxPl1 PxPl2 PxPl3 Px

    Comp Superl Attr Ord Qst IV TV Prt Prs Ind Pot Cond Imprt ImprtII Sg1 Sg2 Sg3 Du1 Du2 Du3 Pl1 Pl2 Pl3 Inf ConNeg Neg PrfPrc VGen PrsPrc Ger Sup Actio VAbess

    Err/Orth

    Semantic tags

    Sem/Act Sem/Ani Sem/Atr Sem/Body Sem/Clth Sem/Domain Sem/Feat-phys Sem/Fem Sem/Group Sem/Lang Sem/Mal Sem/Measr Sem/Money Sem/Obj Sem/Obj-el Sem/Org Sem/Perc-emo Sem/Plc Sem/Sign Sem/State-sick Sem/Sur Sem/Time Sem/Txt

    HUMAN

    PROP-ATTR PROP-SUR

    TIME-N-SET

    Syntactic tags

    @+FAUXV @+FMAINV @-FAUXV @-FMAINV @-FSUBJ> @-F<OBJ @-FOBJ> @-FSPRED<OBJ @-F<ADVL @-FADVL> @-F<SPRED @-F<OPRED @-FSPRED> @-FOPRED> @>ADVL @ADVL< @<ADVL @ADVL> @ADVL @HAB> @<HAB @>N @Interj @N< @>A @P< @>P @HNOUN @INTERJ @>Num @Pron< @>Pron @Num< @OBJ @<OBJ @OBJ> @OPRED @<OPRED @OPRED> @PCLE @COMP-CS< @SPRED @<SPRED @SPRED> @SUBJ @<SUBJ @SUBJ> SUBJ SPRED OPRED @PPRED @APP @APP-N< @APP-Pron< @APP>Pron @APP-Num< @APP-ADVL< @VOC @CVP @CNP OBJ

    -OTHERS SYN-V @X ### Sets containing sets of lists and tags This part of the file lists a large number of sets based partly upon the tags defined above, and partly upon lexemes drawn from the lexicon. See the sourcefile itself to inspect the sets, what follows here is an overview of the set types. #### Sets for Single-word sets INITIAL #### Sets for word or not WORD NOT-COMMA #### Case sets ADLVCASE CASE-AGREEMENT CASE NOT-NOM NOT-GEN NOT-ACC #### Verb sets NOT-V #### Sets for finiteness and mood REAL-NEG MOOD-V NOT-PRFPRC #### Sets for person SG1-V SG2-V SG3-V DU1-V DU2-V DU3-V PL1-V PL2-V PL3-V #### Pronoun sets #### Adjectival sets and their complements #### Adverbial sets and their complements #### Sets of elements with common syntactic behaviour #### NP sets defined according to their morphosyntactic features #### The PRE-NP-HEAD family of sets These sets model noun phrases (NPs). The idea is to first define whatever can occur in front of the head of the NP, and thereafter negate that with the expression **WORD - premodifiers**. #### Border sets and their complements #### Grammarchecker sets * * * This (part of) documentation was generated from [tools/grammarcheckers/grammarchecker.cg3](https://github.com/giellalt/lang-yrk/blob/main/tools/grammarcheckers/grammarchecker.cg3) --- ## tools-tokenisers-tokeniser-disamb-gt-desc.pmscript.md ## Tokeniser for yrk Usage: ``` $ make $ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://github.com/hfst/hfst/wiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1. unknown word-like forms, and 2. unmatched strings We want to give 1) a match, but let 2) be treated specially by `hfst-tokenise -a` Unknowns are made of: * lower-case ASCII * upper-case ASCII * select extended latin symbols * yrk cyrillics ASCII digits * select symbols * Combining diacritics as individual symbols, * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" ### Unknown handling Unknowns are tagged ?? and treated specially with `hfst-tokenise` hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Finally we mark as a token any sequence making up a: * known word in context * unknown (OOV) token in context * sequence of word and punctuation * URL in context * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-disamb-gt-desc.pmscript](https://github.com/giellalt/lang-yrk/blob/main/tools/tokenisers/tokeniser-disamb-gt-desc.pmscript) --- ## tools-tokenisers-tokeniser-gramcheck-gt-desc.pmscript.md ## Grammar checker tokenisation for yrk Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc) Then just: ``` $ make $ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` More usage examples: ``` $ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://github.com/hfst/hfst/wiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1) unknown word-like forms, and 2) unmatched strings We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a * select extended latin symbols * select symbols * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" TODO: Could use something like this, but built-in's don't include šžđčŋ: Simply give an empty reading when something is unknown: hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Finally we mark as a token any sequence making up a: * known word in context * unknown (OOV) token in context * sequence of word and punctuation * URL in context * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript](https://github.com/giellalt/lang-yrk/blob/main/tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript) --- ## tools-tokenisers-tokeniser-tts-cggt-desc.pmscript.md ## TTS tokenisation for smj Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc) Then just: ```sh make echo "ja, ja" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` More usage examples: ```sh echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa \ boasttu olmmoš, man mielde lahtuid." \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst echo "márffibiillagáffe" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://kitwiki.csc.fi/twiki/bin/view/KitWiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1) unknown word-like forms, and 2) unmatched strings We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a * select extended latin symbols * select symbols * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" TODO: Could use something like this, but built-in's don't include šžđčŋ: Simply give an empty reading when something is unknown: hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Needs hfst-tokenise to output things differently depending on the tag they get * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-tts-cggt-desc.pmscript](https://github.com/giellalt/lang-yrk/blob/main/tools/tokenisers/tokeniser-tts-cggt-desc.pmscript)

    Sitemap