What is the intuition of the role of multiple layers in a sequence modelling RNN? If I build a deep sequence modelling recurrent neural-network, for instance a character-level model of text, is there an intuition as to what kinds of features are extracted by the different hidden layers of neurons?
For instance, if I'm using a deep convolutional network for vision, the usual intuition is that the first layer extracts things like edges and corners, the next layer combinations of those, and so on with more and more abstracted concepts as you move up the network.
Is there a similar intuition for RNNs doing text-modelling? The recurrent architecture makes it seem to me that it's not similar to the vision case (i.e. the first layer detecting common letter pairs, the next layer words, the next layer phrase snippets, etc.)
 A: In general, the intuition of multiple layers is to create higher-level abstractions and capture more non-linearities between the data. A meaningful explanation (at least for humans), like the one you described for image processing, is usually achieved with convolutional networks.
Now, in sequence modeling, we want to create such abstractions, and at the same time enforce their correlation with the previous inputs. So, the lower-level abstractions to feedback to the next lower-level abstractions, and so forth and so on with the higher ones. Often the input and output vectors have the same length, where in such cases we just manage to capture more non-linearities between the data (and maybe take advantage of the memory cells in LSTM units).
In the case of NPL, it varies for character level modeling and word level modeling, and I am not sure that you could say something like that.
Similar intuition as yours could be achieved combining convolutions with RNNs, a great example is the work of Kalchbrenner et al..
Finally, a very good blog post, by A. Karpathy, describing character level processing with RNNs might be of your interest.
A: I feel the intution for an RNN will be the function which gets generated includes the vectors previously seen, so any time we feed in the starting sequence the function can give a probablity distribution for next outputs. 
