#SpeakEasy-AI

SpeakEasy-AI is a machine learning project that aims to detect patterns in conversational responses. A fully-trained model is hosted here as an interactive chatbot.

##The Data##

Marvin is trained on data from Reddit comments. The original dataset can be found here, but since Reddit is insane corner of the internet, this data was heavily filtered to correct grammar and exclude long rants/comments containing uncommon words (Marvin's vocabulary is the 25,000 most common words in the entire dataset). Because Reddit allows users to reply directly to another comment, comments are matched with their parent comment to form a prompt/response pair used for training. If a comment has multiple children, the highest-voted one is used.

##The Model##

Our model is an embedding Seq2Seq model built using Google's Tensorflow library and is based initially on their French-English translation model. It is made of LSTM cells, which have an internal cell state that changes as inputs (in our case, words in a sentence) are fed sequentially into the model. This cell state allows the model to consider the context in which an input is recieved, and the output for a given input depends partially on the inputs that came before. Our model has 25,000 input and output nodes (one for each word in the vocabulary) and 3 hidden layers of 768 nodes each.

##Try It Out##

If you have a conversational dataset you want to try out, or if you just want to tinker around, the code in this repo has been written so you can easily play around with model parameters and learning patterns to design your own version of a SpeakEasy chatbot.

###Getting Started###

After cloning the repo, run $ scripts/install.sh from the root directory to configure Tensorflow and set up the virual env. During this process, you will be asked if you want to configure Tensorflow for GPU. Enabling GPU allows for significantly faster training but also requires CUDA/cuDUNN (you have to register as a developer to download cuDUNN) and a NVIDIA graphics card.

###Training Your Model###

Run $ venv/bin/python speak_easy.py to train your model. Running this program without --decode reads reads training and validation data files from the directory specified as --data_dir, builds out a vocabulary based on --vocab size, and tokenizes data by line. The vocabulary and tokenized data are saved to disk after they are built, so data parsing only needs to occur once for a given vocabulary size (see data_utils.py for more information). Training data is used to train a speakEasy model, and validation data is used to evaluate the perplexity of the model at each time-step. Checkpoints are saved to the --train_dir directory every --steps_per_checkpoint, and event logs are saved to --log_dir and can be visualized using Tensorboard.
Running with --decode starts an interactive loop that allows you to interact with the chatbot based on the most recent checkpoint saved in --train_dir. Please note that to decode or to resume training from a previous checkpoint, the parameters describing the architecture of the model must be identical to the ones previously specified during training. Also, it appears that you can only decode using CPU-only tensorflow (I'm not sure why, but running it with GPU gives a memory error) even if the model was previously trained on a GPU-enabled machine.

Several model parameters can be customized using flags. All flag arguments are optional since reasonable default values are provided in runtime_vaiables.py (except --data_dir, whose default needs to be overwritten to point to a data-containing directory):

#####Training parmeters:

#####Model architecture:

#####Data parameters:

#####Directories:

#####Testing: