German language model documentation

All doc-comment documentation in one large file.

---

src-cg3-functions.cg3.md

• Sets for POS sub-categories

• Sets for Semantic tags

• Sets for Morphosyntactic properties

• Sets for verbs

• V is all readings with a V tag in them, REAL-V should be the ones without an N tag following the V.
  The REAL-V set thus awaits a fix to the preprocess V … N bug.

• The set COPULAS is for predicative constructions

• NP sets defined according to their morphosyntactic features

• The PRE-NP-HEAD family of sets

These sets model noun phrases (NPs). The idea is to first define whatever can occur in front of the head of the NP, and thereafter negate that with the expression WORD - premodifiers.

The set NOT-NPMOD is used to find barriers between NPs. Typical usage: … (*1 N BARRIER NPT-NPMOD) … meaning: Scan to the first noun, ignoring anything that can be part of the noun phrase of that noun (i.e., “scan to the next NP head”)

• Miscellaneous sets

• Border sets and their complements

• Syntactic sets

These were the set types.

HABITIVE MAPPING

• hab1

• hab2

• hab3 ( @ADVL>) for hab-actor and hab-case; if leat to the right, and Nom to the right of leat. Lots of restrictions.

• habNomLeft

• hab4

• hab6

• hab7

• hab8 This is not HAB

• hab5 This is not HAB

• habDain ( @ADVL>) for (Pron Dem Pl Loc) if leat followed by Nom to the right

• habGen ( @<ADVL) hab for Gen; if Gen is located in the end of the sentence and Nom is sentence initial

• spred<obj (@SPRED<OBJ) for Acc; the object of an SPRPED. Not to be mistaken with OPRED. If SPRED is to the left, and copulas is to the left of it. Nom or Hab are found sentence initially.

• Hab<spred (@<SPRED) for Nom; if copulas, goallut or jápmit is FMAINV and habitive or human Loc is found to the left. OR: if Ill or @Pron< followed by HAB are found to the left.

• Hab>Advlcase<spred ( @<SUBJ) for Nom; it allows adverbials with Ill/Loc/Com/Ess to be found inbetween HAB and .

• Nom>Advlcase<spred ( @<SUBJ) for Nom; it allows adverbials with Ill/Loc/Com/Ess to be found inbetween Nom and @<SUBJ.

• <spred ( @<SUBJ) for Nom; if copulas to the left, and some kind of adverb, N Loc, time related word or Po to the left of it. OR: if Ill or @Pron< to the left, followed by copulas and the before mentioned to the left of copulas.

• <spred ( @<SUBJ) for Nom, but not for Pers. To the left boahtit or heaŋgát as MAINV, and futher to the left is some kind of place related word, or time related word

• <spredQst1 ( @<SUBJ) for Nom in a typically question sentence; if A) Hab, some kind of place word, Po or Nom to the left, and Qst followed by copulas to the left. B) same as a, only the Qst-pcle is attached to copulas. C) Qst to the left, with copulas to its left, but not if two Nom:s are found somewhere to the right. D) copulas to the left, and BOS to the left. E) Loc or Ill to the left, and Loc or Hab to the left of this, Qst and copulas to the left. F) Num @>N to the left, Hab, some kind of place word, Po or Nom to the left, and Qst followed by copulas to the left. NOTE) for all these rules; human, Loc or Sem/Plc not allowed to the right.

• <spredQst2 (@<SPRED) for Nom; in a typically question sentence; differs from <spredQst1 by not beeing as restricted to the right. Though you are not allowed to be Pers or human.

• Nom<spredQst (@<SPRED) for Nom; in a typically question sentence. Differs from <spredQst2 by letting Nom be found between SPRED and copulas

• <spred (@<SPRED) for A Nom or N Nom if; the subject Nom is on the same side of copulas as you: on the right side of copulas

• <spredVeara (@<SPRED) for veara + Nom; if genitive immediately to the right, and intransitive mainverb to the right of genitive

• leftCop<spred (@<SPRED) for Nom; if copulas is the main verb to the left, and there is no Ess found to the left of cop (note that Loc is allowed between target and cop). OR: if you are Coll or Sem/Group with copulas to your left.

• <spredLocEXPERIMENT (@<SPRED) for material Loc; if you are to the right of copulas, and the Nom to the left of copulas is not a hab-actor

• NumTime (@<SPRED) for A Nom

• <spredSg (@<SPRED) for Sg Nom

• <spredPg (@<SPRED) for Pl Nom

• <spred (@<SPRED) for Nom; if copulas to the left, and Nom or sentence boundary to the left of copulas. First one to the right is EOS.

• <spred (@<SPRED) for N Ess

• spredEss> (@SPRED>) for N Ess; if copulas to the right of you, and if an NP with nom-case first one to your left.

• HABSpredSg> (@SPRED>) for Nom; if habitive first one to the left, followed by copulas.

• GalleSpred> (@SPRED>) for Num Nom; if sentence initial

• spredSgMII> (@SPRED>)

• r492> (@SPRED>) for Interr Gen; consisting only of negations. You are not allowed to be MII. You are not allowed to have an adjective or noun to yor right. You are not allowed to have a verb to your right; the exception beeing an aux.

• AdjSpredSg> (@SPRED>) for A Sg Nom; if copulas to the right, but not if A or @<SPRED are found to the right of copulas

• SpredSg>Hab (@SPRED>) for Nom; if you are sentence initial, copulas is located to the right, and there is a habitive to the right of copulas

• Spred>SubjInf (@SPRED>) for Nom; if copulas to the right, and the subject of copulas is an Inf to the right

• spredCoord (@<SPRED) coordination for Nom; only if there already is a SPRED to the left of CNP. Not if there is some kind of comparison involved.

• subj>Sgnr1 (@SUBJ>) for Nom Sg, including Indef Nom if; VFIN + Sg3 or Pl3 to the right (VFIN not allowed to the left)

• subj>Du (@SUBJ>) for dual nominatives, including Coll Nom. VFIN + Du3 to the right.

• subj>Pl (@SUBJ>) for plural nominatives, including Coll and Sem/Group. VFIN + Pl3 to the right.

• subj>Pl (@SUBJ>) for plural nominatives

• subj>Sgnr2 (@SUBJ>) for Nom Sg; if VFIN + Sg3 to the right.

• <subjSg (@<SUBJ) for Nom Sg; if VFIN Sg3 or Du2 to the left (no HAB allowed to the left).

• f<advl (@-F<ADVL) for infinite adverbials

• f<advl (@-F<ADVL) for infinite adverbials

• s-boundary=advl> (@ADVL>) for ADVL that resemble s-booundaries. Mainverb to the right.

• -fobj> (@-FOBJ>) for Acc

• -fobj> (@-FOBJ>) for Acc

• advl>mainV (@ADVL>) if; finite mainverb not found to the left, but the finite mainverb is found to the right.

• <advl (@<ADVL) if; finite mainverb found to the left. Not if a comma is found immediately to the left and a finite mainverb is located somewhere to the right of this comma.

• <advlPoPr (@<ADVL) if mainverb to the left.

• advlPoPr> (@<ADVL) if mainverb to the right.

• advlEss> (@<ADVL) for weather and time Ess, if FMAINV to the left.

• advl>inbetween (@ADVL>) for Adv; if inbetween two sentenceboundaries where no mainverb is present.

• comma<advlEOS (@<ADVL) if; comma found to the left and the finite mainverb to the left of comma. To the right is the end of the sentence.

• advlBOS> (@ADVL>) if; you are N Ill and found sentnece initially. First one to your right is a clause.

• <advlPoEOS (@<ADVL) for Po; if you are found at the very end of a sentence. A mainverb is needed to the right though.

• cleanupILL<advl (@<ADVL) for N Ill if; there are no boundarysymbols to your left, if you arent already @N< OR @APP-N<, and no mainverb is to yor left.

• <opredAAcc (@<OPRED) for A Acc; if an other accusative to the left, and a transtive verb to the left of it. OR: if a transitive verb to the left, and an accusative to the left of it.

sma object

• <advlEss (@<ADVL) for ESS-ADVL if; FMAINV to the left
• <spredEss (@<SPRED) for N Ess if; FMAINV to the left is intransitive or bargat

SUBJ MAPPING - leftovers

OBJ MAPPING - leftovers

HNOUN MAPPING

---

This (part of) documentation was generated from src/cg3/functions.cg3

---

src-fst-morphology-affixes-adjectives.lexc.md

Adjective inflection The GERMAN language adjectives compare.

• MSC-PRFPRC_BEAMTER ; Beamter:beamte

• FEM-PRFPRC_ANGESTELLTE ; see also Angestellte__n_f

• LEXICON A_/viel__adj viel:

• LEXICON A_minde/r__adj minder:minde

• LEXICON A_n/ah__adj nah:na

---

This (part of) documentation was generated from src/fst/morphology/affixes/adjectives.lexc

---

src-fst-morphology-affixes-adverbs.lexc.md

Adverbs The GERMAN language adverbs

---

This (part of) documentation was generated from src/fst/morphology/affixes/adverbs.lexc

---

src-fst-morphology-affixes-nouns.lexc.md

Boden

Bruder, Apfel

CHECKME

these are from Apertium

N_MSC_PL-with-umlaut

These nouns were previously given a dummy Abhilfe__n_f to indicate they tended to be singular -heit, -keit

This new also in SG Abhilfe__n_f

• LEXICON Abhilfe__n_f This was used in Apertium as FEM SG-only

N_F_PL-in-s

mit Artikel ohne Artikel

• LEXICON Beamte/r__n_m Beamter:beamte

N_Neu_PL-in-umlaut-plus-er

N_Msc_PL-in-umlaut-plus-e

N_Msc_PL-in-umlaut

Let’s check this out Is this just H/andelSG__n_m

the same as K/alb__n_nt

• LEXICON Kapl/an__n_m Kaplan:kaplan

same as K/uh__n_f

THis needs some tending to

• LEXICON Absurd/um__n_nt Absurdum:Absurd

• LEXICON Agend/a__n_f Agenda:Agend

• LEXICON Alb/um__n_nt Album:Alb

• LEXICON Algorithm/us__n_m Algorithmus:Algorithm

• LEXICON Allegr/o__n_nt Allegro:Allegro

• LEXICON Anan/as__n_f Ananas:Ananas

• LEXICON Arabisch__n_nt Arabisch:Arabisch

• LEXICON Arom/a__n_nt Aroma: Arom

• LEXICON Atla/s__n_m Atlas:Atla

• LEXICON Ausdr/uck__n_m

• LEXICON Bab/y__n_nt Baby:Bab

• LEXICON Bas/is__n_f Basis:Bas

• LEXICON Bass/o__n_m Basso:Basso

• LEXICON Bau__n_m

• LEXICON Bronchiti/s__n_f Bronchitis:Bronchiti

• LEXICON Business__n_nt

• LEXICON Unverständnis__n_nt Kein Mehrzahl

• LEXICON Camp/us__n_m Campus:Camp

• LEXICON Cell/o__n_nt Cello:Cello

• LEXICON Cembal/o__n_nt Cembalo:Cembalo

• LEXICON Chairm/an__n_m Chairman:Chairman

• LEXICON Charm/e__n_m Charme:Charm

• LEXICON Cod/ex__n_m Codex:Cod

• LEXICON Cont/e__n_m Conte:cont

• LEXICON Corp/us__n_nt Corpus:corp

• LEXICON Cyberspac/e__n_m Cyberspace:cyberspac

• LEXICON Dem/os__n_m Demos:dem

• LEXICON Disk/us__n_m Diskus:disk

• LEXICON Donn/a__n_f Donna:donn

• LEXICON Ep/os__n_nt Epos:ep

• LEXICON Espress/o__n_m Espresso:espresso

• LEXICON Exam/en__n_nt Examen:exam

• LEXICON Federvie/h__n_nt Federvieh:Federvie

• LEXICON Gefolgs/mann__n_m Gefolgsmann:GEfolgs

• LEXICON Gentlem/an__n_m Gentleman:Gentleman

• LEXICON Hand/ling__n_nt Handling

• LEXICON Hero/s__n_m Heros:Hero

• LEXICON Herz__n_nt

• LEXICON Johanniter__n_f

• LEXICON Kakt/us__n_m Kaktus:Kakt

• LEXICON Kauf/mann__n_m Kaufmann:Kauf

• LEXICON Klima__n_nt

• LEXICON Kod/ex__n_m Kodex:Kod

• LEXICON Kont/o__n_nt Konto:kont

• LEXICON Kon/us__n_m Konus:kon

• LEXICON Korp/us__n_nt Korpus:korp

• LEXICON Kurs/us__n_m Kursus:kurs

• LEXICON Lad/y__n_f Lady:lad

• LEXICON Laudatio__n_f

• LEXICON Le/u__n_m Leu:le

• LEXICON Lexik/on__n_nt Lexikon:lexik

• LEXICON Le/x__n_f Lex:le

• LEXICON Lim/es__n_f Limes:Lim

• LEXICON Lir/a__n_f Lira:Lir

• LEXICON Logo/s__n_m Logos:Logo

• LEXICON Mafios/o__n_m Mafioso:Mafioso

• LEXICON Mark__n_f

• LEXICON Matri/x__n_f Matrix:Matri

• LEXICON M/utter__n_f

• LEXICON Myth/os__n_m Mythos:myth

• LEXICON Op/us__n_nt Opus:op

• LEXICON Paradigm/a__n_nt

• LEXICON Pat/er__n_m Pater:pater

• LEXICON Pen/ny__n_m Penny:Pen

• LEXICON Phalan/x__n_f Phalanx:Phalan

• LEXICON Piazz/a__n_f Piazza:Piazz

• LEXICON Plas/tik__n_nt

• LEXICON Präs/es__n_m Präses:Präs

• LEXICON Requie/m__n_nt Requiem:Requie

• LEXICON Re/x__n_m Rex:re

• LEXICON Ris/iko__n_nt Risiko:Ris

• LEXICON Rowd/y__n_m Rowdy:Rowd

• LEXICON Sald/o__n_m Saldo:saldo

• LEXICON Spektr/um__n_nt Spektrum:Spektr

• LEXICON Stadi/on__n_nt Stadion:Stadio

• LEXICON Städteatla/s__n_m Städteatlas:Städteatla

• LEXICON Tr/opf__n_m

• LEXICON UFO__n_nt

• LEXICON Vektor__n_m Vektor:vektor

• LEXICON Vir/us__n_nt Virus:Vir

• LEXICON Volum/en__n_nt Volumen:volum

• LEXICON Walr/oss__n_nt Walross:walross

• LEXICON Xylophon__n_nt Xylophon:xylophon

• LEXICON Kirmes__n_f

CHECKME ʼ CHECKME ʼ

---

This (part of) documentation was generated from src/fst/morphology/affixes/nouns.lexc

---

src-fst-morphology-affixes-prepositions.lexc.md

Prepositions The GERMAN language prepositions

Not definite

---

This (part of) documentation was generated from src/fst/morphology/affixes/prepositions.lexc

---

src-fst-morphology-affixes-propernouns.lexc.md

Proper noun inflection The GERMAN language proper nouns inflect in the same cases as regular nouns, but with a colon (‘:’) as separator.

CHECKME ʼ

CHECKME ʼ

CHECKME ʼ

CHECKME ʼ

FIXME CHECKME ʼ FIXME CHECKME ʼ

CHECKME ʼ FIXME CHECKME ʼ

CHECKME ʼ FIXME CHECKME ʼ CHECKME ʼ

FIXME

CHECKME ʼ FIXME CHECKME ʼ CHECKME ʼ

CHECKME ʼ FIXME CHECKME ʼ CHECKME ʼ

---

This (part of) documentation was generated from src/fst/morphology/affixes/propernouns.lexc

---

src-fst-morphology-affixes-symbols.lexc.md

Symbol affixes

---

This (part of) documentation was generated from src/fst/morphology/affixes/symbols.lexc

---

src-fst-morphology-affixes-verbs.lexc.md

CHECKME, should there be +Comp and +Superl

CHECKME, should there be +Comp and +Superl

• +Ind+Prs+Sg1:%^Pen%^RmVow%>e POST-V_01 ; works with ackern

• LEXICON V_haben__vbhaver haben:ha

• LEXICON V_/müssen__vbmod müssen:muss

• LEXICON V_/dürfen__vbmod dürfen:darf

• LEXICON V_/können__vbmod können:konn

• LEXICON V_/mögen__vbmod mögen:mog

• LEXICON V_/sollen__vbmod sollen:soll

• LEXICON V_/werden__vaux werden:w

• LEXICON V_/wollen__vaux wollen:w

• LEXICON V_/werden__vblex werden:w

• LEXICON V_/kommen__vblex kommen:k

• LEXICON V_g/eben__vblex geben:gab

• LEXICON V_spr/echen__vblex geben:gab

• LEXICON V_g/elten__vblex gelten:galt

• LEXICON V_tr/eten__vblex treten:tratt

• LEXICON V_g/ehen__vblex gehen:g

• LEXICON V_st/ehen__vblex stehen:st

• LEXICON V_s/ehen__vblex sehen:sah

• LEXICON V_schr/ecken__vblex schrecken:schrack

• LEXICON V_w/erfen__vblex werfen:warf

• LEXICON V_b/ersten__vblex bersten:barst birst

• LEXICON V_k/ennen__vblex kennen:kann

• LEXICON V_s/enden__vblex senden:sand

• LEXICON V_/nehmen__vblex nehmen:n

• LEXICON V_dr/eschen__vblex dreschen:drosch

• LEXICON V_/laufen__vblex laufen:l

• LEXICON V_h/alten__vblex halten:halt

• LEXICON V_l/assen__vblex lassen:l

• LEXICON V_st/oßen__vblex stoßen:stoß

• LEXICON V_bl/eiben__vblex bleiben:bleib

• LEXICON V_r/eißen__vblex reißen:reiss

• LEXICON V_gl/eichen__vblex gleichen:gleich

• LEXICON V_schw/immen__vblex schwimmen:schwamm

• LEXICON V_br/ingen__vblex bringen:br

• LEXICON V_d/ingen__vblex dingen:dang

• LEXICON V_f/inden__vblex finden:fand

• LEXICON V_s/itzen__vblex sitzen:s

• LEXICON V_d/enken__vblex denken:d

• LEXICON V_bew/egen__vblex bewegen:bew

• LEXICON V_pfl/egen__vblex pflegen:pfl

• LEXICON V_z/iehen__vblex ziehen:z

• LEXICON V_fr/ieren__vblex frieren:fror

• LEXICON V_sch/ießen__vblex schießen:schoss

• LEXICON V_f/ahren__vblex fahren:fahr

• LEXICON V_f/angen__vblex fangen:fang

• LEXICON V_/dürfen__vblex dürfen:darf

• LEXICON V_l/iegen__vblex liegen:lag

---

This (part of) documentation was generated from src/fst/morphology/affixes/verbs.lexc

---

src-fst-morphology-phonology.twolc.md

TRIGGER

• %^A2E:0
• %^A2I:0
• %^A2O:0
• %^A2Ö:0
• %^A2U:0
• %^A2Ü:0
• %^A2IE:0
• %^O2E:0
• %^O2I:0
• %^O2IE:0
• %^EI2IE:0
• %^EI2I:0
• %^CC2C:0 mm:m0 kommen kommst kamst
• %^DeV:0 g0:ch
• %^LowerVow:0 o:a
• %^Pen:0 penultimate syllable
• %^RmVow:0 remove vowel

• %^Umlaut:0 umlaut a:ä, o:ö, u:ü

• Boden%^Umlaut%>#

• Böden000

• ★Boden%^Umlaut%># (is not standard language)

• ★Boden000 (is not standard language)

• konn%^LowerVow

• kann0

• Baum%^Umlaut%>e#

• Bäum00e0

• Apfel%^Umlaut

• Äpfel0

• Napf%^Umlaut%>e#

• Näpf00e0

• ★Napf%^Umlaut%>e# (is not standard language)
• ★Napf00e0 (is not standard language)

dreschen+V+Ind+Prs+Pl1

• drosch%^O2E%>en
• dresch0%>en

geben+V+Ind+Prs+Sg1

• gab%^A2E%>e
• geb0%>e

geben+V+Ind+Prs+Sg2

• gab%^A2I%>st
• gib0%>st halten+V+Ind+Prt+Sg2
• ha0lt%^A2IE%>e%>st
• hielt0%>e%>st fallen+V+Ind+Prt+Pl1
• fa0ll%^A2IE%^CC2C%>en
• fiel00%>en

schwimmen+V+Ind+Prs+Sg2

• schwamm%^A2I%>st
• schwimm0%>st

fahren+V+Ind+Prt+Sg2

• fahr%^A2U%>st
• fuhr0%>st

fahren+V+Ind+Prt+Sg2

• fahr%^A2Ü%>st
• führ0%>st

gelten+V+Subj+Prt+Sg1

• galt%^A2Ö%>e
• gölt0%>e

stoßen+V+Ind+Prt+Sg2

• sto0ß%^O2IE%>e%>st
• stieß0%>e%>st dreschen+V+Ind+Prs+Sg3
• drosch%^O2I%>t

• drisch0%>t

• Saal%^Umlaut%>e#
• Sä0l00e0

Salto

• Salto%^RmVow
• Salt00

reißen

• reiss%^CC2C%>en
• rei0ß0%>en

treffen+V+Subj+Prt+Pl1

• traff%^Umlaut%^CC2C%>en
• trä0f00%>en

aasen+V+Ind+Prs+Sg2

• aas%>st
• aas%>0t groß+A+Superl+Fem+Sg+Acc+Def
• groß%^Umlaut%>st%>e
• größ0%>0t%>e

reißen

• reiss%^CC2C%>en
• rei0ß0%>en

fallen+V+Ind+Prt+Pl1

• fa0ll%^A2IE%^CC2C%>en
• fie0l0%>en

schrecken+V+Subj+Prt+Pl1

• schrack%^Umlaut%^CC2C%>en

• schrä0k00%>en

• Nomade%>en#

• Nomad00en0

• ★Nomade%>en# (is not standard language)
• ★Nomade0en0 (is not standard language) ackern+V+Ind+Prs+Sg1
• acker%^Pen%^RmVow%>e
• ack0r00%>e bleiben+V+Ind+Prt+Sg1
• blei0b%^EI2IE
• bl0ieb0 reißen
• reiss%^EI2I
• r0iss0

mögen+V+Mod+Ind+Prt+Sg1

• mog0%^DeV%>te
• moch0%>te

mögen+V+Mod+Ind+Prt+Sg1

• mog0%^DeV%>te
• moch0%>te

stoßen+V+Ind+Prt+Sg2

• sto0ß%^O2IE%>e%>st
• stieß0%>e%>st halten+V+Ind+Prt+Sg2
• ha0lt%^A2IE%>e%>st
• hielt0%>e%>st bleiben+V+Ind+Prt+Sg1
• blei0b%^EI2IE
• bl0ieb0

---

This (part of) documentation was generated from src/fst/morphology/phonology.twolc

---

src-fst-morphology-root.lexc.md

German language

• +SP2 singular or plural Formal

• +NoArt Stark accompanied preceding Indef Schwach Def gemischt Indef

• +Symbol = independent symbols in the text stream, like £, €, ©

Homonyms

Semantic tags to help disambiguation & synt. analysis: (before POS)

• +Sem/Act = Activity
• +Sem/Adr = Webadr
• +Sem/Amount = Amount
• +Sem/Ani = Animate
• +Sem/Aniprod = Animal Product
• +Sem/Body = Bodypart
• +Sem/Body-abstr = siellu, vuoig?a, jierbmi
• +Sem/Build = Building
• +Sem/Build-part = Part of Bulding, like the closet
• +Sem/Cat = Category
• +Sem/Clth = Clothes
• +Sem/Clth-jewl = Jewelery
• +Sem/Clth-part = part of clothes, boallu, sávdnji…
• +Sem/Ctain = Container
• +Sem/Ctain-abstr = Abstract container like bank account
• +Sem/Ctain-clth = Soft container, like a rucksack
• +Sem/Curr = Currency like dollár, Not Money
• +Sem/Dance = Dance
• +Sem/Dir = Direction like GPS-kursa
• +Sem/Domain = Domain like politics, reindeerherding (a system of actions)
• +Sem/Drink = Drink
• +Sem/Dummytag = Dummytag
• +Sem/Edu = Educational event
• +Sem/Event = Event
• +Sem/Feat = Feature, like Árvu
• +Sem/Feat-phys = Physiological feature, ivdni, fárda
• +Sem/Feat-psych = Psychological feauture
• +Sem/Feat-measr = Psychological feauture
• +Sem/Fem = Female name
• +Sem/Food = Food
• +Sem/Food-med = Medicine
• +Sem/Furn = Furniture
• +Sem/Game = Game
• +Sem/Geom = Geometrical object
• +Sem/Group = Animal or Human Group
• +Sem/Hum = Human
• +Sem/Hum-abstr = Human abstract
• +Sem/Ideol = Ideology
• +Sem/Lang = Language
• +Sem/Mal = Male name
• +Sem/Mat = Material for producing things
• +Sem/Measr = Measure
• +Sem/Money = Has to do with money, like wages, not Curr(ency)
• +Sem/Obj = Object
• +Sem/Obj-clo = Cloth
• +Sem/Obj-cogn = Cloth
• +Sem/Obj-el = (Electrical) machine or apparatus
• +Sem/Obj-ling = Object with something written on it
• +Sem/Obj-rope = flexible ropelike object
• +Sem/Obj-surfc = Surface object
• +Sem/Org = Organisation
• +Sem/Part = Feature, oassi, bealli
• +Sem/Perc-cogn = Cloth
• +Sem/Perc-emo = Emotional perception
• +Sem/Perc-phys = Physical perception
• +Sem/Perc-psych = Physical perception
• +Sem/Plant = Plant
• +Sem/Plant-part = Plant part
• +Sem/Plc = Place
• +Sem/Plc-abstr = Abstract place
• +Sem/Plc-elevate = Place
• +Sem/Plc-line = Place
• +Sem/Plc-water = Place
• +Sem/Pos = Position (as in social position job)
• +Sem/Process = Process
• +Sem/Prod = Product
• +Sem/Prod-audio = Audio product
• +Sem/Prod-cogn = Cognition product
• +Sem/Prod-ling = Linguistic product
• +Sem/Prod-vis = Visual product
• +Sem/Rel = Relation
• +Sem/Route = Route
• +Sem/Rule = Rule or convention
• +Sem/Semcon = Semantic concept
• +Sem/Sign = Sign (e.g. numbers, punctuation)
• +Sem/Sport = Sport
• +Sem/State =
• +Sem/State-sick = Illness
• +Sem/Substnc = Substance, like Air and Water
• +Sem/Sur = Surname
• +Sem/Symbol = Symbol
• +Sem/Time = Time
• +Sem/Tool = Prototypical tool for repairing things
• +Sem/Tool-catch = Tool used for catching (e.g. fish)
• +Sem/Tool-clean = Tool used for cleaning
• +Sem/Tool-it = Tool used in IT
• +Sem/Tool-measr = Tool used for measuring
• +Sem/Tool-music = Music instrument
• +Sem/Tool-write = Writing tool
• +Sem/Txt = Text (girji, lávlla…)
• +Sem/Veh = Vehicle
• +Sem/Wpn = Weapon
• +Sem/Wthr = The Weather or the state of ground

Multiple Semantic tags:

• +Sem/Act_Group Activity and Group
• +Sem/Act_Plc A persons job is an activity, and a place as well
• +Sem/Act_Route Activity and Route, ie johtolat
• +Sem/Amount_Build Amount and Building
• +Sem/Amount_Semcon
• +Sem/Ani_Body-abstr_Hum
• +Sem/Ani_Build
• +Sem/Ani_Build-part
• +Sem/Ani_Build_Hum_Txt
• +Sem/Ani_Group
• +Sem/Ani_Group_Hum
• +Sem/Ani_Hum
• +Sem/Ani_Hum_Plc
• +Sem/Ani_Hum_Time
• +Sem/Ani_Plc
• +Sem/Ani_Plc_Txt
• +Sem/Ani_Time
• +Sem/Ani_Veh
• +Sem/Aniprod_Hum
• +Sem/Aniprod_Obj-clo
• +Sem/Aniprod_Perc-phys
• +Sem/Aniprod_Plc
• +Sem/Ant_Fem Anthtroponym Female
• +Sem/Ant_Mal Anthtroponym Male
• +Sem/Body-abstr_Prod-audio_Semcon
• +Sem/Body_Body-abstr
• +Sem/Body_Clth
• +Sem/Body_Food
• +Sem/Body_Group_Hum
• +Sem/Body_Hum
• +Sem/Body_Mat
• +Sem/Body_Measr
• +Sem/Body_Obj_Tool-catch
• +Sem/Body_Plc
• +Sem/Body_Time
• +Sem/Build-part_Plc
• +Sem/Build_Build-part
• +Sem/Build_Clth-part
• +Sem/Build_Edu_Org
• +Sem/Build_Event_Org
• +Sem/Build_Org
• +Sem/Build_Route
• +Sem/Clth-jewl_Curr
• +Sem/Clth-jewl_Money
• +Sem/Clth-jewl_Plant
• +Sem/Clth_Hum
• +Sem/Ctain-abstr_Org
• +Sem/Ctain-clth_Plant
• +Sem/Ctain-clth_Veh
• +Sem/Ctain_Feat-phys
• +Sem/Ctain_Furn
• +Sem/Ctain_Tool
• +Sem/Ctain_Tool-measr
• +Sem/Curr_Org
• +Sem/Dance_Org
• +Sem/Dance_Prod-audio
• +Sem/Domain_Food-med
• +Sem/Domain_Prod-audio
• +Sem/Edu_Event
• +Sem/Edu_Group_Hum
• +Sem/Edu_Mat
• +Sem/Edu_Org
• +Sem/Event_Food
• +Sem/Event_Hum
• +Sem/Event_Plc
• +Sem/Event_Time
• +Sem/Feat-measr_Plc
• +Sem/Feat-phys_Tool-write
• +Sem/Feat-phys_Veh
• +Sem/Feat-phys_Wthr
• +Sem/Feat-psych_Hum
• +Sem/Feat_Plant
• +Sem/Food_Perc-phys
• +Sem/Food_Plant
• +Sem/Game_Obj-play
• +Sem/Geom_Obj
• +Sem/Group_Hum
• +Sem/Group_Hum_Org
• +Sem/Group_Hum_Plc
• +Sem/Group_Hum_Prod-vis
• +Sem/Group_Org
• +Sem/Group_Sign
• +Sem/Group_Txt
• +Sem/Hum_Lang
• +Sem/Hum_Lang_Plc
• +Sem/Hum_Lang_Time
• +Sem/Hum_Obj
• +Sem/Hum_Org
• +Sem/Hum_Plant
• +Sem/Hum_Plc
• +Sem/Hum_Tool
• +Sem/Hum_Veh
• +Sem/Hum_Wthr
• +Sem/Lang_Tool
• +Sem/Mat_Plant
• +Sem/Mat_Txt
• +Sem/Measr_Time
• +Sem/Money_Obj
• +Sem/Money_Txt
• +Sem/Obj-play
• +Sem/Obj-play_Sport
• +Sem/Obj_Semcon
• +Sem/Clth-jewl_Org
• +Sem/Org_Rule
• +Sem/Org_Txt
• +Sem/Org_Veh
• +Sem/Part_Prod-cogn
• +Sem/Perc-emo_Wthr
• +Sem/Plant_Plant-part
• +Sem/Plant_Tool
• +Sem/Plant_Tool-measr
• +Sem/Plc-abstr_Rel_State
• +Sem/Plc-abstr_Route
• +Sem/Plc_Pos
• +Sem/Plc_Route
• +Sem/Plc_Substnc
• +Sem/Plc_Substnc_Wthr
• +Sem/Plc_Time
• +Sem/Plc_Tool-catch
• +Sem/Plc_Wthr
• +Sem/Prod-audio_Txt
• +Sem/Prod-cogn_Txt
• +Sem/Semcon_Txt
• +Sem/Obj_State
• +Sem/Substnc_Wthr
• +Sem/Time_Wthr

Derivation

Valency tags, i.e. tags assigned to verbs or prepositions for denoting their arguments

• +%<acc%> case tags
• +%<dat%> case tags
• +%<gen%> case tags
• +%<nom%> case tags

TRIGGER

• %^A2E
• %^A2I
• %^A2O
• %^A2Ö
• %^A2U
• %^A2Ü
• %^A2IE
• %^O2E
• %^O2I
• %^O2IE
• %^EI2IE
• %^EI2I
• %^CC2C mm:m0 kommen kommst kamst
• %^DeV g0:ch
• %^LowerVow o:a
• %^Pen penultimate syllable
• %^RmVow remove vowel
• %^Umlaut umlaut a:ä, o:ö, u:ü

Escaped symbols

Symbols that need to be escaped on the lower side (towards twolc):

• »
• «
• <

The Usage extents are marked using following tags:

• +Err/Orth
• ** +Use/Marg ** marginal (?)
• ** +Use/-Spell ** Excluded in speller
• ** +Use/SpellNoSugg ** recognized but not suggested in speller

COMPOUNDING

• +Cmp compound
• +Use/Circ FIXME, (where I am)

Flag diacritics

We have manually optimised the structure of our lexicon using following flag diacritics to restrict morhpological combinatorics - only allow compounds with verbs if the verb is further derived into a noun again: | @P.NeedNoun.ON@ | (Dis)allow compounds with verbs unless nominalised | @D.NeedNoun.ON@ | (Dis)allow compounds with verbs unless nominalised | @C.NeedNoun@ | (Dis)allow compounds with verbs unless nominalised

For languages that allow compounding, the following flag diacritics are needed to control position-based compounding restrictions for nominals. Their use is handled automatically if combined with +CmpN/xxx tags. If not used, they will do no harm. | @P.CmpFrst.FALSE@ | Require that words tagged as such only appear first | @D.CmpPref.TRUE@ | Block such words from entering ENDLEX | @P.CmpPref.FALSE@ | Block these words from making further compounds | @D.CmpLast.TRUE@ | Block such words from entering R | @D.CmpNone.TRUE@ | Combines with the next tag to prohibit compounding | @U.CmpNone.FALSE@ | Combines with the prev tag to prohibit compounding | @P.CmpOnly.TRUE@ | Sets a flag to indicate that the word has passed R | @D.CmpOnly.FALSE@ | Disallow words coming directly from root.

Use the following flag diacritics to control downcasing of derived proper nouns (e.g. Finnish Pariisi -> pariisilainen). See e.g. North Sámi for how to use these flags. There exists a ready-made regex that will do the actual down-casing given the proper use of these flags. | @U.Cap.Obl@ | Allowing downcasing of derived names: deatnulasj. | @U.Cap.Opt@ | Allowing downcasing of derived names: deatnulasj.

• @U.Cap.Obl@ Nouns ; This is stated once since R also leads directly to Nouns

---

This (part of) documentation was generated from src/fst/morphology/root.lexc

---

src-fst-morphology-stems-abbreviations.lexc.md

Work: the -X_ should be developed to convey part of speech, e.g. eigtl.:eigtl. AB-DOT-X_ ; » eigtl.:eigtl AB-DOT-ADV_ ;

FROM APERTIUM

---

This (part of) documentation was generated from src/fst/morphology/stems/abbreviations.lexc

---

src-fst-morphology-stems-adjectives.lexc.md

Adjectives Adjectives in GERMAN language describe things.

FROM APERTIUM

---

This (part of) documentation was generated from src/fst/morphology/stems/adjectives.lexc

---

src-fst-morphology-stems-adverbs.lexc.md

Adverbs Adverbs in GERMAN language describe things.

FROM APERTIUM

---

This (part of) documentation was generated from src/fst/morphology/stems/adverbs.lexc

---

src-fst-morphology-stems-exceptions.lexc.md

Verbs

---

This (part of) documentation was generated from src/fst/morphology/stems/exceptions.lexc

---

src-fst-morphology-stems-interjections.lexc.md

FROM APERTIUM

---

This (part of) documentation was generated from src/fst/morphology/stems/interjections.lexc

---

src-fst-morphology-stems-nouns.lexc.md

• Uhr w2f “(eng) clock” ; Uhr+N+Pl+Nom: Uhren
• Ammer w2f “(eng) bunting” ; Ammer+N+Pl+Nom: Ammern
• Karbonade w2f ; Karbonade+N+Pl+Nom: Karbonaden

FROM APERTIUM and beyond

• Bank+Hom1:bank K/uh__n_f “(eng) bench” ; Bank+N+Pl+Nom: Bänke B/ank__n_f
• Bank+Hom2:bank Abfuhr__n_f “(eng) bank” ; Bank+N+Pl+Nom: Banken 2016-01-25

---

This (part of) documentation was generated from src/fst/morphology/stems/nouns.lexc

---

src-fst-morphology-stems-numerals.lexc.md

Numerals Numerals in GERMAN language are numbers.

• **LEXICON ROMAN ** roman numerals

• **LEXICON ROM-THOUSAND **

• **LEXICON ROM-THOUSAND-TAG **

• **LEXICON ROM-HUNDRED **

• **LEXICON ROM-HUNDRED-TAG **

• **LEXICON ROM-TEN **

• **LEXICON ROM-TEN-TAG **

• **LEXICON ROM-ONE **

• **LEXICON ROM-ONE-TAG **

• **LEXICON ROM-SPLIT **

• **LEXICON 2ROMAN **

• **LEXICON 2ROM-THOUSAND **

• **LEXICON 2ROM-THOUSAND-TAG **

• **LEXICON 2ROM-HUNDRED **

• **LEXICON 2ROM-HUNDRED-TAG **

• **LEXICON 2ROM-TEN **

• **LEXICON 2ROM-TEN-TAG **

• **LEXICON 2ROM-ONE **

• **LEXICON 2ROM-ONE-TAG **

• **LEXICON ROMNUMTAG **

---

This (part of) documentation was generated from src/fst/morphology/stems/numerals.lexc

---

src-fst-morphology-stems-prefixes.lexc.md

Prefixes Prefixes in GERMAN language are bound to beginning of other words.

---

This (part of) documentation was generated from src/fst/morphology/stems/prefixes.lexc

---

src-fst-morphology-stems-propernouns.lexc.md

FROM APERTIUM

---

This (part of) documentation was generated from src/fst/morphology/stems/propernouns.lexc

---

src-fst-morphology-stems-verbs.lexc.md

wiederherangekommen

This is where we put BE/EMP/ENT/ER words that are loners

The verbs below can take BE/ENT/ER/VER/ZER

auxillary and modal verbs other verbs

APERTIUM

---

This (part of) documentation was generated from src/fst/morphology/stems/verbs.lexc

---

src-fst-phonetics-txt2ipa.xfscript.md

retroflex plosive, voiceless t ʈ 0288, 648 ( = ASCII 096) retroflex plosive, voiced d ɖ 0256, 598 labiodental nasal F ɱ 0271, 625 retroflex nasal n ɳ 0273, 627 palatal nasal J ɲ 0272, 626 velar nasal N ŋ 014B, 331 uvular nasal N\ ɴ 0274, 628

bilabial trill B\ ʙ 0299, 665 uvular trill R\ ʀ 0280, 640 alveolar tap 4 ɾ 027E, 638 retroflex flap r ɽ 027D, 637 bilabial fricative, voiceless p\ ɸ 0278, 632 bilabial fricative, voiced B β 03B2, 946 dental fricative, voiceless T θ 03B8, 952 dental fricative, voiced D ð 00F0, 240 postalveolar fricative, voiceless S ʃ 0283, 643 postalveolar fricative, voiced Z ʒ 0292, 658 retroflex fricative, voiceless s ʂ 0282, 642 retroflex fricative, voiced z` ʐ 0290, 656 palatal fricative, voiceless C ç 00E7, 231 palatal fricative, voiced j\ ʝ 029D, 669 velar fricative, voiced G ɣ 0263, 611 uvular fricative, voiceless X χ 03C7, 967 uvular fricative, voiced R ʁ 0281, 641 pharyngeal fricative, voiceless X\ ħ 0127, 295 pharyngeal fricative, voiced ?\ ʕ 0295, 661 glottal fricative, voiced h\ ɦ 0266, 614

alveolar lateral fricative, vl. K alveolar lateral fricative, vd. K\

labiodental approximant P (or v) alveolar approximant r\ retroflex approximant r` velar approximant M\

retroflex lateral approximant l` palatal lateral approximant L velar lateral approximant L
Clicks

bilabial O\ (O = capital letter) dental |
(post)alveolar !\ palatoalveolar =\ alveolar lateral ||
Ejectives, implosives

ejective > e.g. ejective p p> implosive < e.g. implosive b b< Vowels

close back unrounded M close central unrounded 1 close central rounded } lax i I lax y Y lax u U

close-mid front rounded 2 close-mid central unrounded @\ close-mid central rounded 8 close-mid back unrounded 7

schwa ə @

open-mid front unrounded E open-mid front rounded 9 open-mid central unrounded 3 open-mid central rounded 3\ open-mid back unrounded V open-mid back rounded O

ash (ae digraph) { open schwa (turned a) 6

open front rounded & open back unrounded A open back rounded Q Other symbols

voiceless labial-velar fricative W voiced labial-palatal approx. H voiceless epiglottal fricative H\ voiced epiglottal fricative <\ epiglottal plosive >\

alveolo-palatal fricative, vl. s\ alveolo-palatal fricative, voiced z\ alveolar lateral flap l\ simultaneous S and x x\ tie bar _ Suprasegmentals

primary stress “ secondary stress % long : half-long :\ extra-short _X linking mark -
Tones and word accents

level extra high _T level high _H level mid _M level low _L level extra low _B downstep ! upstep ^ (caret, circumflex)

contour, rising contour, falling _F contour, high rising _H_T contour, low rising _B_L

contour, rising-falling _R_F (NB Instead of being written as diacritics with _, all prosodic marks can alternatively be placed in a separate tier, set off by < >, as recommended for the next two symbols.) global rise global fall Diacritics

voiceless 0 (0 = figure), e.g. n_0 voiced _v aspirated _h more rounded _O (O = letter) less rounded _c advanced _+ retracted _- centralized _” syllabic = (or _=) e.g. n= (or n=) non-syllabic _^ rhoticity `

breathy voiced _t creaky voiced _k linguolabial _N labialized _w palatalized ‘ (or _j) e.g. t’ (or t_j) velarized _G pharyngealized _?\

dental d apical _a laminal _m nasalized ~ (or _~) e.g. A~ (or A~) nasal release _n lateral release _l no audible release _}

velarized or pharyngealized _e velarized l, alternatively 5 raised _r lowered _o advanced tongue root _A retracted tongue root _q

---

This (part of) documentation was generated from src/fst/phonetics/txt2ipa.xfscript

---

src-fst-transcriptions-transcriptor-abbrevs2text.lexc.md

We describe here how abbreviations are in German are read out, e.g. for text-to-speech systems.

For example:

• s.:syntynyt # ;
• os.:omaa% sukua # ;
• v.:vuosi # ;
• v.:vuonna # ;
• esim.:esimerkki # ;
• esim.:esimerkiksi # ;

---

This (part of) documentation was generated from src/fst/transcriptions/transcriptor-abbrevs2text.lexc

---

src-fst-transcriptions-transcriptor-numbers-digit2text.lexc.md

% komma% :, Root ; % tjuohkkis% :%. Root ; % kolon% :%: Root ; % sárggis% :%- Root ; % násti% :%* Root ;

---

This (part of) documentation was generated from src/fst/transcriptions/transcriptor-numbers-digit2text.lexc

---

tools-grammarcheckers-grammarchecker.cg3.md

[ L A N G U A G E ] G R A M M A R C H E C K E R

DELIMITERS

TAGS AND SETS

Tags

This section lists all the tags inherited from the fst, and used as tags in the syntactic analysis. The next section, Sets, contains sets defined on the basis of the tags listed here, those set names are not visible in the output.

Beginning and end of sentence

BOS EOS

Parts of speech tags

N A Adv V Pron CS CC CC-CS Po Pr Pcle Num Interj ABBR ACR CLB LEFT RIGHT WEB PPUNCT PUNCT

COMMA ¶

Tags for POS sub-categories

Pers Dem Interr Indef Recipr Refl Rel Coll NomAg Prop Allegro Arab Romertall

Tags for morphosyntactic properties

Nom Acc Gen Ill Loc Com Ess Ess Sg Du Pl Cmp/SplitR Cmp/SgNom Cmp/SgGen Cmp/SgGen PxSg1 PxSg2 PxSg3 PxDu1 PxDu2 PxDu3 PxPl1 PxPl2 PxPl3 Px

Comp Superl Attr Ord Qst IV TV Prt Prs Ind Pot Cond Imprt ImprtII Sg1 Sg2 Sg3 Du1 Du2 Du3 Pl1 Pl2 Pl3 Inf ConNeg Neg PrfPrc VGen PrsPrc Ger Sup Actio VAbess

Err/Orth

Semantic tags

Sem/Act Sem/Ani Sem/Atr Sem/Body Sem/Clth Sem/Domain Sem/Feat-phys Sem/Fem Sem/Group Sem/Lang Sem/Mal Sem/Measr Sem/Money Sem/Obj Sem/Obj-el Sem/Org Sem/Perc-emo Sem/Plc Sem/Sign Sem/State-sick Sem/Sur Sem/Time Sem/Txt

HUMAN

PROP-ATTR PROP-SUR

TIME-N-SET

Syntactic tags

@+FAUXV @+FMAINV @-FAUXV @-FMAINV @-FSUBJ> @-F<OBJ @-FOBJ> @-FSPRED<OBJ @-F<ADVL @-FADVL> @-F<SPRED @-F<OPRED @-FSPRED> @-FOPRED> @>ADVL @ADVL< @<ADVL @ADVL> @ADVL @HAB> @<HAB @>N @Interj @N< @>A @P< @>P @HNOUN @INTERJ @>Num @Pron< @>Pron @Num< @OBJ @<OBJ @OBJ> @OPRED @<OPRED @OPRED> @PCLE @COMP-CS< @SPRED @<SPRED @SPRED> @SUBJ @<SUBJ @SUBJ> SUBJ SPRED OPRED @PPRED @APP @APP-N< @APP-Pron< @APP>Pron @APP-Num< @APP-ADVL< @VOC @CVP @CNP OBJ

-OTHERS SYN-V @X ## Sets containing sets of lists and tags This part of the file lists a large number of sets based partly upon the tags defined above, and partly upon lexemes drawn from the lexicon. See the sourcefile itself to inspect the sets, what follows here is an overview of the set types. ### Sets for Single-word sets INITIAL ### Sets for word or not WORD NOT-COMMA ### Case sets ADLVCASE CASE-AGREEMENT CASE NOT-NOM NOT-GEN NOT-ACC ### Verb sets NOT-V ### Sets for finiteness and mood REAL-NEG MOOD-V NOT-PRFPRC ### Sets for person SG1-V SG2-V SG3-V DU1-V DU2-V DU3-V PL1-V PL2-V PL3-V ### Pronoun sets ### Adjectival sets and their complements ### Adverbial sets and their complements ### Sets of elements with common syntactic behaviour ### NP sets defined according to their morphosyntactic features ### The PRE-NP-HEAD family of sets These sets model noun phrases (NPs). The idea is to first define whatever can occur in front of the head of the NP, and thereafter negate that with the expression **WORD - premodifiers**. ### Border sets and their complements ### Grammarchecker sets * * * This (part of) documentation was generated from [tools/grammarcheckers/grammarchecker.cg3](https://github.com/giellalt/lang-deu/blob/main/tools/grammarcheckers/grammarchecker.cg3) --- # tools-tokenisers-tokeniser-disamb-gt-desc.pmscript.md # Tokeniser for deu Usage: ``` $ make $ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://github.com/hfst/hfst/wiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1. unknown word-like forms, and 2. unmatched strings We want to give 1) a match, but let 2) be treated specially by `hfst-tokenise -a` Unknowns are made of: * lower-case ASCII * upper-case ASCII * select extended latin symbols ASCII digits * select symbols * Combining diacritics as individual symbols, * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" ## Unknown handling Unknowns are tagged ?? and treated specially with `hfst-tokenise` hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Finally we mark as a token any sequence making up a: * known word in context * unknown (OOV) token in context * sequence of word and punctuation * URL in context * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-disamb-gt-desc.pmscript](https://github.com/giellalt/lang-deu/blob/main/tools/tokenisers/tokeniser-disamb-gt-desc.pmscript) --- # tools-tokenisers-tokeniser-gramcheck-gt-desc.pmscript.md # Grammar checker tokenisation for deu Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc) Then just: ``` $ make $ echo "ja, ja" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` More usage examples: ``` $ echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa boasttu olmmoš, man mielde lahtuid." | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst $ echo "márffibiillagáffe" | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://github.com/hfst/hfst/wiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1) unknown word-like forms, and 2) unmatched strings We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a * select extended latin symbols * select symbols * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" TODO: Could use something like this, but built-in's don't include šžđčŋ: Simply give an empty reading when something is unknown: hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Finally we mark as a token any sequence making up a: * known word in context * unknown (OOV) token in context * sequence of word and punctuation * URL in context * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript](https://github.com/giellalt/lang-deu/blob/main/tools/tokenisers/tokeniser-gramcheck-gt-desc.pmscript) --- # tools-tokenisers-tokeniser-tts-cggt-desc.pmscript.md # TTS tokenisation for smj Requires a recent version of HFST (3.10.0 / git revision>=3aecdbc) Then just: ```sh make echo "ja, ja" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` More usage examples: ```sh echo "Juos gorreválggain lea (dárbbašlaš) deavdit gáibádusa \ boasttu olmmoš, man mielde lahtuid." \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst echo "(gáfe) 'ja' ja 3. ja? ц jaja ukjend \"ukjend\"" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst echo "márffibiillagáffe" \ | hfst-tokenise --giella-cg tokeniser-disamb-gt-desc.pmhfst ``` Pmatch documentation: <https://kitwiki.csc.fi/twiki/bin/view/KitWiki/HfstPmatch> Characters which have analyses in the lexicon, but can appear without spaces before/after, that is, with no context conditions, and adjacent to words: * Punct contains ASCII punctuation marks * The symbol after m-dash is soft-hyphen `U+00AD` * The symbol following {•} is byte-order-mark / zero-width no-break space `U+FEFF`. Whitespace contains ASCII white space and the List contains some unicode white space characters * En Quad U+2000 to Zero-Width Joiner U+200d' * Narrow No-Break Space U+202F * Medium Mathematical Space U+205F * Word joiner U+2060 Apart from what's in our morphology, there are 1) unknown word-like forms, and 2) unmatched strings We want to give 1) a match, but let 2) be treated specially by hfst-tokenise -a * select extended latin symbols * select symbols * various symbols from Private area (probably Microsoft), so far: * U+F0B7 for "x in box" TODO: Could use something like this, but built-in's don't include šžđčŋ: Simply give an empty reading when something is unknown: hfst-tokenise --giella-cg will treat such empty analyses as unknowns, and remove empty analyses from other readings. Empty readings are also legal in CG, they get a default baseform equal to the wordform, but no tag to check, so it's safer to let hfst-tokenise handle them. Needs hfst-tokenise to output things differently depending on the tag they get * * * This (part of) documentation was generated from [tools/tokenisers/tokeniser-tts-cggt-desc.pmscript](https://github.com/giellalt/lang-deu/blob/main/tools/tokenisers/tokeniser-tts-cggt-desc.pmscript)