Recently, I have been looking at seq2seq models that have been used for translating from one language to another using recurrent neural networks (often with LSTM cells).
Those models can also be used to generate text, one character at a time. Based on its internal memory, which efficiently encodes the previous characters, the model learns a probability distribution for the next character.
When looking at the various implementations of these seq2seq models, like this one, I see an embedding matrix is trained jointly with the neural network. As I understand it, each of this matrix's rows is the 'embedding' of a particular character (each character is represented by an integer: its id in a finite vocabulary).
What is the rationale behind using this embedding? What is it used for? Why is it needed?
LSTM: Long-Short Term Memory