An open-source test tool for rule-based FST moprhological analysers. morph-test is at the core of the morphological testing regime in the GiellaLT infrastructure.
Clone or download and run as shown below.
python3 morph-test.py -h
usage: morph-test.py [-h] [-c] [-o OUTPUT] [-q] [-i] [-s] [-l] [-f] [-p] [-S [SECTION]]
[-t [TEST]] [-F [TRANSDUCER]] [-v] [--app [APP]]
[--gen [GEN]] [--morph [MORPH]]
test_file
Test morphological transducers for consistency.
positional arguments:
test_file YAML file with test rules
optional arguments:
-h, --help show this help message and exit
-c, --colour Colours the output
-o OUTPUT, --output OUTPUT
Desired output style: compact, terse, final, normal (Default: normal)
-q, --silent Hide all output; exit code only
-i, --ignore-extra-analyses
Ignore extra analyses when there are more than expected, will PASS
if the expected one is found.
-s, --surface Surface input/analysis tests only
-l, --lexical Lexical input/generation tests only
-f, --hide-fails Suppresses fails to make finding passes easier
-p, --hide-passes Suppresses passes to make finding fails easier
-S [SECTION], --section [SECTION]
The section to be used for testing (default is `hfst`)
-t [TEST], --test [TEST]
Which test to run (Default: all). TEST = test ID, e.g.
'Noun - gåetie' (remember quotes if the ID contains spaces)
-F [TRANSDUCER], --fallback [TRANSDUCER]
Which fallback transducer to use (ignored, use --gen and --morph).
-v, --verbose More verbose output.
--app [APP] Override application used for test
--gen [GEN] Override generation transducer used for test
--morph [MORPH] Override morph transducer used for test
Will run all tests in the test_file by default.
The different output modes (-o OUTPUT, --output OUTPUT) will render the as follows:
- normal (default): all details, full report:
-----------------------------------------------------
Test 0: Adjective - antisosiálla (Lexical/Generation)
-----------------------------------------------------
[1/7][PASS] antisosiálla+A+Attr => antisosiála
[1/7][PASS] antisosiálla+A+Attr => antisosiálalasj
[2/7][PASS] antisosiálla+A+Sg+Nom => antisosiálla
[2/7][PASS] antisosiálla+A+Sg+Nom => antisosiálalasj
[3/7][PASS] antisosiálla+A+Sg+Gen => antisosiála
[3/7][PASS] antisosiálla+A+Sg+Gen => antisosiálalattja
[4/7][PASS] antisosiálla+A+Der/Comp+A+Sg+Nom => antisosiálalabbo
[4/7][PASS] antisosiálla+A+Der/Comp+A+Sg+Nom => antisosiálap
[5/7][PASS] antisosiálla+A+Der/Superl+A+Sg+Nom => antisosiálamus
[5/7][PASS] antisosiálla+A+Der/Superl+A+Sg+Nom => antisosiálalamos
[6/7][PASS] antisosiálla+A+Der/AAdv+Adv => antisosiálalattjat
[7/7][PASS] antisosiálla+A+Attr+Der/vuota+N+Sg+Nom => <No lexical/generation>
Test 0 - Passes: 12, Fails: 0, Total: 12
----------------------------------------------------
Test 1: Adjective - aspektuálla (Lexical/Generation)
----------------------------------------------------
[1/7][PASS] aspektuælla+A+Attr => aspektuella
[1/7][PASS] aspektuælla+A+Attr => aspektuellalasj
[1/7][FAIL] aspektuælla+A+Attr => Unexpected results: aspektuälla
[2/7][PASS] aspektuælla+A+Sg+Nom => aspektuælla
[2/7][PASS] aspektuælla+A+Sg+Nom => aspektuälla
[2/7][PASS] aspektuælla+A+Sg+Nom => aspektuellalasj
[3/7][PASS] aspektuælla+A+Sg+Acc => aspektuellav
[3/7][PASS] aspektuælla+A+Sg+Acc => aspektuellalattjav
[3/7][FAIL] aspektuælla+A+Sg+Acc => Unexpected results: aspektuällav
[4/7][PASS] aspektuælla+A+Der/Comp+A+Sg+Nom => aspektuellalabbo
[4/7][PASS] aspektuælla+A+Der/Comp+A+Sg+Nom => aspektuellap
[4/7][FAIL] aspektuælla+A+Der/Comp+A+Sg+Nom => Unexpected results: aspektuällap
[5/7][PASS] aspektuælla+A+Der/Superl+A+Sg+Nom => aspektuellamus
[5/7][PASS] aspektuælla+A+Der/Superl+A+Sg+Nom => aspektuellalamos
[5/7][FAIL] aspektuælla+A+Der/Superl+A+Sg+Nom => Unexpected results: aspektuällamus
[6/7][PASS] aspektuælla+A+Der/AAdv+Adv => aspektuellalattjat
[7/7][PASS] aspektuælla+A+Attr+Der/vuota+N+Sg+Nom => <No lexical/generation>
Test 1 - Passes: 13, Fails: 4, Total: 17
--------------------------------------------------
Test 2: Adjective - paralælla (Lexical/Generation)
--------------------------------------------------
[1/5][PASS] paralælla+A+Attr => paralellalasj
[1/5][PASS] paralælla+A+Attr => paralella
[1/5][FAIL] paralælla+A+Attr => Unexpected results: paralälla
[2/5][PASS] paralælla+A+Sg+Nom => paralälla
[2/5][PASS] paralælla+A+Sg+Nom => paralellalasj
[2/5][PASS] paralælla+A+Sg+Nom => paralælla
[3/5][PASS] paralælla+A+Sg+Gen => paralellalattja
[3/5][PASS] paralælla+A+Sg+Gen => paralella
[3/5][FAIL] paralælla+A+Sg+Gen => Unexpected results: paralälla
[4/5][PASS] paralælla+A+Der/Comp+A+Sg+Nom => paralellap
[4/5][PASS] paralælla+A+Der/Comp+A+Sg+Nom => paralellalabbo
[4/5][FAIL] paralælla+A+Der/Comp+A+Sg+Nom => Unexpected results: paralällap
[5/5][PASS] paralælla+A+Der/Superl+A+Sg+Nom => paralellamus
[5/5][PASS] paralælla+A+Der/Superl+A+Sg+Nom => paralellalamos
[5/5][FAIL] paralælla+A+Der/Superl+A+Sg+Nom => Unexpected results: paralällamus
Test 2 - Passes: 11, Fails: 4, Total: 15
---------------------------------------------------
Test 3: Adjective - antisosiálla (Surface/Analysis)
---------------------------------------------------
[1/9][PASS] antisosiála => antisosiálla+A+Attr
[1/9][PASS] antisosiála => antisosiálla+A+Sg+Gen
[2/9][PASS] antisosiálalasj => antisosiálla+A+Attr
[2/9][PASS] antisosiálalasj => antisosiálla+A+Sg+Nom
[3/9][PASS] antisosiálla => antisosiálla+A+Sg+Nom
[4/9][PASS] antisosiálalattja => antisosiálla+A+Sg+Gen
[5/9][PASS] antisosiálalabbo => antisosiálla+A+Der/Comp+A+Sg+Nom
[6/9][PASS] antisosiálap => antisosiálla+A+Der/Comp+A+Sg+Nom
[7/9][PASS] antisosiálalamos => antisosiálla+A+Der/Superl+A+Sg+Nom
[8/9][PASS] antisosiálamus => antisosiálla+A+Der/Superl+A+Sg+Nom
[9/9][PASS] antisosiálalattjat => antisosiálla+A+Der/AAdv+Adv
Test 3 - Passes: 11, Fails: 0, Total: 11
--------------------------------------------------
Test 4: Adjective - aspektuálla (Surface/Analysis)
--------------------------------------------------
[ 1/11][PASS] aspektuella => aspektuælla+A+Attr
[ 2/11][PASS] aspektuellalasj => aspektuælla+A+Attr
[ 2/11][PASS] aspektuellalasj => aspektuælla+A+Sg+Nom
[ 3/11][PASS] aspektuælla => aspektuælla+A+Sg+Nom
[ 4/11][PASS] aspektuälla => aspektuælla+A+Sg+Nom
[ 5/11][PASS] aspektuellav => aspektuælla+A+Sg+Acc
[ 6/11][PASS] aspektuellalattjav => aspektuælla+A+Sg+Acc
[ 7/11][PASS] aspektuellalabbo => aspektuælla+A+Der/Comp+A+Sg+Nom
[ 8/11][PASS] aspektuellap => aspektuælla+A+Der/Comp+A+Sg+Nom
[ 9/11][PASS] aspektuellalamos => aspektuælla+A+Der/Superl+A+Sg+Nom
[10/11][PASS] aspektuellamus => aspektuælla+A+Der/Superl+A+Sg+Nom
[11/11][PASS] aspektuellalattjat => aspektuælla+A+Der/AAdv+Adv
Test 4 - Passes: 12, Fails: 0, Total: 12
------------------------------------------------
Test 5: Adjective - paralælla (Surface/Analysis)
------------------------------------------------
[1/9][PASS] paralella => paralælla+A+Sg+Gen
[1/9][PASS] paralella => paralælla+A+Attr
[2/9][PASS] paralellalasj => paralælla+A+Sg+Nom
[2/9][PASS] paralellalasj => paralælla+A+Attr
[3/9][PASS] paralælla => paralælla+A+Sg+Nom
[4/9][PASS] paralälla => paralælla+A+Sg+Nom
[5/9][PASS] paralellalattja => paralælla+A+Sg+Gen
[6/9][PASS] paralellalabbo => paralælla+A+Der/Comp+A+Sg+Nom
[7/9][PASS] paralellap => paralælla+A+Der/Comp+A+Sg+Nom
[8/9][PASS] paralellalamos => paralælla+A+Der/Superl+A+Sg+Nom
[9/9][PASS] paralellamus => paralælla+A+Der/Superl+A+Sg+Nom
Test 5 - Passes: 11, Fails: 0, Total: 11
Total passes: 70, Total fails: 8, Total: 78
- compact: Report the test names and result for each, and print a summary:
[PASS] Test 0: Adjective - antisosiálla (Lexical/Generation) 12/0/12
[FAIL] Test 1: Adjective - aspektuálla (Lexical/Generation) 13/4/17
[FAIL] Test 2: Adjective - paralælla (Lexical/Generation) 11/4/15
[PASS] Test 3: Adjective - antisosiálla (Surface/Analysis) 11/0/11
[PASS] Test 4: Adjective - aspektuálla (Surface/Analysis) 12/0/12
[PASS] Test 5: Adjective - paralælla (Surface/Analysis) 11/0/11
Total passes: 70, Total fails: 8, Total: 78
- terse: print one
.for each PASS, and one!for each fail, and report PASS or FAIL:
............
..!.....!..!..!..
..!.....!..!..!
...........
............
...........
FAIL
- final: only prints a summary of PASSed/FAILed/TOTAL tests:
70/8/78
Verbose mode (-v, --verbose) adds test run metadata:
`/usr/local/bin/hfst-optimized-lookup` will be used for parsing dictionaries.
Generating...
Morphing...
Done!
[... test results ...]
Silent or quiet mode (-q, --silent ) blocks all output, and only returns the exit value (0 for PASS, 1 for FAIL).
-f, --hide-fails removes the fails from the output, to make finding passes easier.
-p, --hide-passes removes passes from the output, to make finding fails easier.
The test files are written in YAML syntax, adhering to the following structure:
Config:
hfst:
Gen: ../../../src/generator-gt-norm.hfst
Morph: ../../../src/analyser-gt-norm.hfst
xerox:
Gen: ../../../src/generator-gt-norm.xfst
Morph: ../../../src/analyser-gt-norm.xfst
App: lookup
Tests:
Pronoun - juoga: # Indefinitive
juoga+Pron+Indef+Sg+Nom: juoga
juoga+Pron+Indef+Sg+Gen: juoŋga
juoga+Pron+Indef+Sg+Acc: juojddá
juoga+Pron+Indef+Sg+Ine: [juonná, juonne]
juoga+Pron+Indef+Sg+Ill: juosik
juoga+Pron+Indef+Sg+Ela: juosstá
juoga+Pron+Indef+Sg+Com: juojnak
juoga+Pron+Indef+Pl+Nom: []
juoga+Pron+Indef+Pl+Gen: []
juoga+Pron+Indef+Pl+Acc: juojddájt
juoga+Pron+Indef+Pl+Ine: []
juoga+Pron+Indef+Pl+Ill: []
juoga+Pron+Indef+Pl+Ela: []
juoga+Pron+Indef+Pl+Com: []
juoga+Pron+Indef+Ess: juonenThe Config section contains defaults to be used with the test files; these can be overriden on the command line. After the Config section comes the Tests section. Each test must have a name (Pronoun - juoga in the example above), followed by the actual test data.
Each line of test data contains:
- morphological analysis: e.g.
juoga+Pron+Indef+Sg+Gen - separator:
: - expected word form(s):
juoŋga
If more than one word form is expected, they must be in a [] list, comma separated.
If no word form is expected, that must be denoted with an empty [].
Comments can be freely added, marked with an initial #.
A test run succeeds if all and only the listed word forms are generated, and the word forms listed only get the specified analyses.
All languages contain a certain amount of homonymy, which makes the -i, --ignore-extra-analyses option very useful: it makes the tests pass even if there are alternative analyses of a given word form. That is, homonym analyses won't destroy the test results.
Licensed under CC0 1.0 Universal.
Fork and PR on Github.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you shall be licensed as above, without any additional terms or conditions.