1
$\begingroup$

Let's say I'm given a stream of characters, one at a time. Some example strings of characters that could occur might be "01234" or "hello how are". They might even be encodings of waveforms as integers from 0-255 encoded into ascii.

My goal is to train a model to predict the next character in the sequence (or more than one character if possible). Each character generated is drawn from a distribution that may be dependent on any number of previous things I have seen, but not necessarily adversarialy.

How well can I do/how can I do this?

I don't think there is one best approach and am generally just looking for methods I can try so multiple different answers are fine.The three methods I am aware of are recurrent neural nets, (potentially hidden) markov chains, and prediction of the next item using a traditional classifier on a fixed window. These have their own benefits and drawbacks (which might be nice to discuss here), but what else is there?

$\endgroup$
  • $\begingroup$ Is the problem you're working on the generic prediction of any arbitrary data type? Your example sequences are numeric, natural language and DSP, which are each their own specialization. I'm not sure anyone has sufficiently abstracted the problem of "what comes next?" to any input, free of context. $\endgroup$ – Sycorax Jan 17 '17 at 23:09
  • $\begingroup$ Sorta, I'm really just looking for techniques that one can use for solving some subset of these kinds of problems. $\endgroup$ – Phylliida Jan 18 '17 at 0:46
1
$\begingroup$

The OpenNMT project at Harvard is an open-source toolkit for NLP machine learning based on sequential prediction of markov chain processes. They have a github repository here ... https://github.com/OpenNMT and some documentation here ... https://arxiv.org/abs/1701.02810

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.