- A GAN is a Generative Adversarial Network.
- They use Neural Network architechure to generate all kinds of things such as images and text!
- GANs do not have labels or targets therefore they are a unsupervised machine learning technique.
- Once trained on text data or images they are ableto generate something that could come close to being in the original dataset.
- I used two different GAN architectures to generate my text.
- Both architectures were trained on the same individual character South Park script data
- The first architecture I used was a simple LSTM neural network.
- The second was a retrained simple-GPT2
- What is GPT2?
- GPT2 is an open source text generative model that was made and trained by OPENAI.
- Using a google collab notebook with some premade opensource code I was able to further train a GPT2 model on my South Park characters.
- To tackle this question I built a discriminator that used the generated text from both models and the real script data and trained to predict whether the text given was real or generated!
- I used a Random Forest classifier because it is quite fast at trainging since it can work in parallel, and it is quite accurate without much need for tuning.
- The GPT2 generated text was able to be distinguished roughly 60% of the time vs. the LSTM generated text was able to be distinguished 100% of the time.
- This shows that the GPT2 model was able to generate more realistic text since the accuracy of the random forest classifier was lower than that of the LSTM model.
- GPT2 training collab notebook: https://colab.research.google.com/drive/10AcSRJi1fW5DKMugJZ0qGmnc_rBOHVLW#scrollTo=VHdTL8NDbAh3