(科技大擂台 與AI對話)

• A CNTK (Microsoft deep learning toolkit) implementation of CS565600 competition
• We use LSTM + attention to do this task
• For more model information, please refer to the report
• If you meet any problem in this repo, feel free to contact zlsh80826@gmail.com

Here are some required libraries for training

• python3
• cuda-9.0 (CNTK required)
• openmpi-1.10 (CNTK required)
• gcc >= 6 (CNTK required)

• Please refer requirements.txt

We recommand you to run all the scripts in script directory

cd AppropriateResponsePrediction/script

Each script contain helper, you can check it for customed settings.

python <some script>.py --help

This script will convert the text format program to processed npy format.

You can specify --threads to indicate how many threads you want to use.

python preprocessing.py

This script will train the Traditional Chinese Embedding with processed data.

python train_fasttext.py

Default settings will generate 4 million training data, which will consume about 8 GB disk space.

python gen_training.py

CNTK support large training file, but we need to convert it to ctf format.

python tsv2ctf.py

Default settings will run 300 epochs and save the checkpoint of each epoch.

python train.py

Inference script will read the checkpoint and do the inference. So you can inference while training, 4 - 10 epochs result is good enough in my experimence.

python inference.py

Based on the Kaggle Leaderboard, our implementation is second prize (the first two are fake).