Utilities for development polish language support for Spacy.

Guide is in this document.

Use separated folders for every module pipeline.

We store all artifacts in folders data and models.

To add new step to pipeline use

dvc run [-d <dependencies>] [-o <result_artifacts>] (-f <step_name>.dvc) <command>

in main folder.

When something in pipeline changed and you want to run all steps again, use

dvc repro <step_name>

with <step_name> to which you want to rerun pipeline.

First, set up remote.

Synchronization always works with in connection with current branch and commit. dvc push Send all tracked artifacts to remote. dvc pull Get all tracked arfitfacts from remote.

dvc pipeline show -c --ascii

To package model please run python deployment/combine_and_package.py with proper arguments. For example

python deployment/combine_and_package.py \
 --pos-path models/pos_NKJP_word2vec/model-fial/ \
 --tree-path models/trees-pos_LFG_word2vec/model-final/ \
 --ner-path models/ner_nkjp_word2vec/model-final/ \
 --output-path models/release \
 --blank-vectors-path models/blank_NKJP_word2vec/

fill poll presented by script with usefull infomations