ugh I can't remember the name of that cleaning up book so let's make all the varieties using word2vec to find synonyms of the words in the title
and other books too, obviously.
see @WhatToWriteNext
This uses word2vec, which is some existence-altering wizardry of its own. Tutorial and explanation below -- along with an interactive word2vec model: http://radimrehurek.com/2014/02/word2vec-tutorial/
- use an off-the-shelf stemmer and compare the removed portion (leaf?) in the word and synonym for levenshtein distance
- pre-parse (by hand) into SimpleNLG structures then synonymize each word (using an API) then use NLG for conjugation
- what about parsing with NLTK and combining the POS and the word ("Vtrash", "NNtrash"), and then using those as the input to the model?
http://developer.wordnik.com/docs.html#!/word/getRelatedWords_get_4
TODO: consider shrinking the window size to take advantage of more syntactic similarity:
Effect of Window Size The size of the sliding window has a strong effect on the re- sulting vector similarities. Larger windows tend to produce more topical similarities (i.e. \dog", \bark" and \leash" will be grouped together, as well as \walked", \run" and \walk- ing"), while smaller windows tend to produce more functional and syntactic similarities (i.e. \Poodle", \Pitbull", \Rottweiler", or \walking",\running",\approaching"). from http://u.cs.biu.ac.il/~yogo/nnlp.pdf, p23 "Positional windows" also useful (rather than training on "the", train on "the:-2" if the focus word occurs two words after "the" )
Themed book titles "What if Harry Potter was about artists"
this requires a list of titles ALONG WITH their topics so we can do something like:
Harry Potter and the Chamber of Secrets minus magic plus art and see what we get.
I'm using a corpus of sentences from The New York Times that I, unfortunately, can't share. Here are some alternatives that'd probably work fine:
- pretrained Google News model: https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit
- and from the University of Leipzig: eng_news_2013_3M.tar.gz