./src/extract_socialiqa.py downloads the socialiqa dataset, extracts it, and prepares it for the training of the smart filter classifier.

./src/filter_inference.py runs the smart filters trained in ./src/filter_training.py to filter the data.

For the download of the Books corpus, please refer to https://github.com/soskek/bookcorpus for the moment. This will change.