Inspired by the Harry Potter: Written by Artificial Intelligence article that was going around a few days ago, I wanted to try my hand at creating a similar system.
I've taken the Coursera Machine Learning course, so I have a (very!) basic understanding of the concepts involved in Neural Networks, but haven't implemented anything beyond the examples in that course.
My planned approach was to first train a Neural Network to recognize samples of my own writing (with the features/inputs being the presence of trigrams of words, and the ouput being a 0-1 value encoding the network's confidence that the example is a sample of my writing), then to use a genetic approach to generate more and more feasible sentences: generate a random string of words, generate multiple "children" by randomly mutating a word or two at a time, select those that the NN scores highest, and repeat.
However, this approach seems flawed due to the huge number of features involved.
nltk.corpus.words.words has size 236736, so the set of all trigrams of words is of size
236736**3~=10**16 - which I imagine will render my operations time-consuming, to say the least (I recall from the ML course that it's a design smell for models to have more than 100,000 features). I could try to limit the trigrams to only those constructed from words that I've previously used - but, even with a severely-restricted vocabulary of 1000 words, I would have a billion features.
I was trying to write this while on a plane (and without internet), but I see now that there's a linked guide. Still, I'd like some feedback on my original approach.
Is my general approach (train NN to recognize my writing, use NN as fitness measure for a genetic algorithm) sound, and I just need to select features differently? Or does is this entire strategy infeasible and I should approach this problem a different way?