This is the official Python implementation of WordCraft: An Environment for Benchmarking Commonsense Agents.
If you use this code in your work, please cite our paper:
@inproceedings{jiang2020wordcraft,
title={Learning to communicate with deep multi-agent reinforcement learning},
author={Minqi Jiang and Jelena Luketina and Nantas Nardelli and Pasquale Minervini and Philip H.S. Torr and Shimon Whiteson and Tim Rockt\"{a}schel},
booktitle={Workshop on Language in Reinforcement Learning (LaRel)},
year={2020},
url={https://github.com/minqi/wordcraft},
}
Create a conda environment and install dependencies:
conda env create -f environment.yml
conda activate wordcraft
pip install -r requirements.txt
Install spaCy model for GloVe embeddings:
python -m spacy download en_core_web_lg
Train a self-attention actor-critic using IMPALA on WordCraft:
python -m run \
--xpid "testing" \
--env 'textlab-multistep-goal-v0' \
--split by_recipe \
--train_ratio 0.8 \
--depths 1 \
--num_distractors 1 \
--feature_type glove \
--discounting 0.99 \
--reward_clipping none \
--learning_rate 0.001 \
--entropy_cost 0.01 \
--unroll_length 2 \
--total_steps 3000000 \
--arch selfattn \
--num_actors 2 \
--num_threads 4 \
--batch_size 128 \
--test_interval 50000 \
--num_test_episodes 500 \
--verbose
Create a new WordCraft Gym environment: env = gym.make('textlab-multistep-goal-v0', **kwargs). See the table below for a listing of the relevant keyword arguments.
| Argument | Description | default |
|---|---|---|
| data_path | relative path to recipes .json | datasets/alchemy2.json |
| recipe_book_path | absolute path to a serialized RecipeBook object, useful for ensuring train/test split consistency | None |
| feature_type | How words are represented. One of ['one_hot', 'random', 'glove'] | 'glove' |
| shuffle_features | Set to True to shuffle GloVe features w.r.t. words | False |
| random_feature_size | Size of random features | 300 |
| max_depth | Task depth, the number of recipe combinations needed to reach the goal | 1 |
| split | How to split train and test sets. One of ['by_goal', 'by_recipe'] | 'by_recipe' |
| train_ratio | Ratio of goals or recipe used for training | 1.0 |
| num_distractors | Number of irrelevant items present in the starting set | 0 |
| subgoal_rewards | Set to True to reward creating intermediate entities and penalize, irrelevant entities | True |
| seed | RNG seed, though see N.B. below for ensuring truly reproducible behavior. | None |
Try running the environment in interactive mode :)
python -m scripts.interactive --depth=1 --num_distractors=1 --max_mix_steps=1
Note --max_mix_steps is the total number of recipe attempts allowed. It defaults to the task depth.
To make environments completely deterministic, you must first fix PYTHONHASHSEED in the shell by executing export PYTHONHASHSEED="0".
You can turn random hashing back on by setting this value back to random: export PYTHONHASHSEED="random".