Grammarchecker testing

The grammarchecker.cg3 ruleset may be tested in a number of ways.

Regression testing

Building test file for regression testing

In lang-$LANG/tools/grammarchecker/tests, add files testfile.yaml with the following format:

Config:
  Spec: ../pipespec.xml
  Variant: smngram-dev

Tests:
  - "Muu šiljoost {lii}£{láá} kyehti rástágáá."

Note: The lines following “Config:” and “Tests:” start with two spaces. The testsentences are quoted. The error marking could be more detailed, it is documented in the principles of error marking document.

Testing the whole set of test files

Overview

The test are run by standing in tools/grammarchecker and writing make check. This will give a report on whether the tests passed or failed.

Overview in tabular form

for i in tools/grammarcheckers/tests/*.yaml; do echo $(basename $i) $(gtgramtool test -c yaml -o final $i 2>/dev/null); done

The output may then be turned into a table or whatever.

Testing file by file

To get a more detailed report for each sentence in the different files, do the following, where FILENAME.yaml is the name of the file you want to test (change sme to your relevant language):

gtgramtool test -c yaml $GTLANGS/lang-sme/tools/grammarcheckers/tests/FILENAME.yaml

If you have a file with the -PASS.yaml suffix that suddenly fails, hide the passes to easier see what fails:

gtgramtool test -c -p yaml $GTLANGS/lang-sme/tools/grammarcheckers/tests/FILENAME-PASS.yaml

Testing for false alarms

Run your hopefully correct text (here called mainly_correct_textfile.txt) through the grammarchecker (here: smn, standing in lang-smn), and inspect the result for false alarms.

cat mainly_correct_textfile.txt | sh tools/grammarchecker/modes/trace-smngram.mode > misc/xx_falsealarmcheck.txt

These sentences may also be extracted and formatted for regression testing, se below.

Building a corpus of positives (false alarm candidates)

This is documented in the file Extracting sentences for precision testing.