Alex Graves created a model to generate hand writing sequences which use an LSTM (kind of Recurrent Neural Network) to predict the parameters for an mixture model. The mixture model is then used to predict the next x,y coordinate and whether the pen is up or down.

Generating Sequences With Recurrent Neural Networks

Demo of the above

Why not just use the LSTM to directly predict the pen offsets (x,Y) and up/down flag. Why bother with the mixture model?

  • 5
    $\begingroup$ you could send an email and ask Alex I really don't know the answer. If I were doing this, I would use a mixture model if there multiple modes. In character recognition each character is a single mode. All lower-case "c" letters belong in the same stack. $\endgroup$ Jan 23, 2016 at 15:25
  • $\begingroup$ I changed the title from "Why does Alex Graves use a mixture model with his RNN instead of just directly predictive real values?" In principle, there is nothing to stop Alex Graves answering the question himself, but particularly after 2 years of being posted, the more general question in the revised title seems more suitable for CV. (I have no idea what the answer is!) $\endgroup$
    – Nick Cox
    Jun 30, 2017 at 9:13
  • $\begingroup$ Do we have an answer yet? Or has anyone tried to remove the mixture model? $\endgroup$
    – shouldsee
    Feb 9, 2022 at 15:41

1 Answer 1


I am very new in the area so please take my comments with care !

I saw a lecture from Alex in this youtube channel https://www.youtube.com/channel/UC0z_jCi0XWqI8awUuQRFnyw

In this video, the first objective of Alex was to build a simulation model. Therefore, I think that the output of the model which is searched for is a random variable.

With this, every time you ask the model a simulation, you will have different results (unless you set the variance to 0 as he does in the end of the presentation under the "biases samples").

I also think that if you do have direct output (such as actually telling move the pen right 10 pixels), since in the structure (both inputs and NN structure) there is no stochastic components you could end with the same results in all cases.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.