This repository contains the source code based on the paper A Fast Unified Model for Sentence Parsing and Understanding and original codebase. For a more informal introduction to the ideas behind the model, see this Stanford NLP blog post.

The included implementations are:

• A Python/Pytorch implementation of SPINN using a naïve stack representation (named fat-stack)

The Python code lives, quite intuitively, in the python folder. We used this code to train and test the SPINN models before publication.

Requirements:

• Python 2.7
• Pytorch

Install most required Python dependencies using the command below.

pip install -r python/requirements.txt

Install Pytorch based on instructions online: http://pytorch.org

The main executable for the SNLI experiments in the paper is supervised_classifier.py, whose flags specify the hyperparameters of the model. You can specify gpu usage by setting --gpu flag greater than or equal to 0. Uses the CPU by default.

Here's a sample command that runs a fast, low-dimensional CPU training run, training and testing only on the dev set. It assumes that you have a copy of SNLI available locally.

PYTHONPATH=spinn/python \
    python2.7 -m spinn.models.supervised_classifier --data_type nli \
    --training_data_path ~/data/snli_1.0/snli_1.0_dev.jsonl \
    --eval_data_path ~/data/snli_1.0/snli_1.0_dev.jsonl \
    --embedding_data_path python/spinn/tests/test_embedding_matrix.5d.txt \
    --word_embedding_dim 5 --model_dim 10 --model_type CBOW

For full runs, you'll also need a copy of the 840B word 300D GloVe word vectors.

You can train SPINN using only sentence-level labels. In this case, the integrated parser will randomly sample labels during training time, and will be optimized with the REINFORCE algorithm. The command to run this model looks slightly different:

PYTHONPATH=spinn/python \
    python2.7 -m spinn.models.rl_classifier --data_type listops \
    --training_data_path spinn/python/spinn/data/listops/train_d20a.tsv \
    --eval_data_path spinn/python/spinn/data/listops/test_d20a.tsv  \
    --word_embedding_dim 32 --model_dim 32 --mlp_dim 16 --model_type RLSPINN \
    --rl_baseline value --rl_reward standard --rl_weight 42.0

Note: This model does not yet work well on natural language data, although it does on the included synthetic dataset called listops. Please look at the sweep file for an idea of which hyperparameters to use.

This project contains a handful of tools for easier analysis of your model's performance.

For one, after a periodic number of batches, some useful statistics are printed to a file specified by --log_path. This is convenient for visual inspection, and the script parse_logs.py is an example of how to easily parse this log file.

In addition, there is support for realtime summaries using Visdom. This requires a few steps:

1. Run your experiment normally, but specify a --metrics_path.
2. Run Visdom in it's own terminal instance: python -m visdom.server
3. Run this project's visdom_reporter.py script, specifying a root which matches the --metrics_path flag: python scripts/visdom_reporter.py --root $METRICS_PATH

Then open Visdom in a browser window to see graphs representing accuracy, loss and some other metrics updated in real time. This is most useful when running multiple experiments simultaneously.

If you're interested in proposing a change or fix to SPINN, please submit a Pull Request. In addition, ensure that existing tests pass, and add new tests as you see appropriate. To run tests, simply run this command from the root directory:

nosetests python/spinn/tests

SPINN outputs metrics and statistics into a text protocol buffer format. When adding new fields to the proto file, the generated proto code needs to be updated.

bash python/build.sh

Copyright 2017, New York University

Licensed under the Apache License, Version 2.0 (the "License"); you may not use these files except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.