Keras - LSTM: need for a final dense layer I am looking at the text generation example using Keras here and I noticed that a Dense(len(chars)) is included as the last layer. 
Given that LSTM itself can predict the next character directly, why is there a need for a final dense layer? Instead, why can't I simply do:
model.add(LSTM(len(chars), return_sequences=False))
 A: You can certainly try to do that, i.e. remove the final fully-connected (dense) layer
model.add(LSTM(512, return_sequences=False))
# model.add(Dropout(0.2))
# model.add(Dense(len(chars)))
# model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='rmsprop')

but you're going to have to accommodate for the Input dimension mis-match. 
ValueError: Input dimension mis-match. (input[0].shape[1] = 512, input[3].shape[1] = 57)
If you set model.add(LSTM(512, return_sequences=True)) you'll get the right size, but theano compilation will fail:
TypeError: ('Bad input argument to theano function at index 1(0-based)', 'Wrong number of dimensions: expected 3, got 2 with shape (128, 57).')
The final Dense layer is meant to be an output layer with softmax activation, allowing for 57-way classification of the input vectors. 
Look at all the Keras LSTM examples, during training, backpropagation-through-time starts at the output layer, so it serves an important purpose with your chosen optimizer=rmsprop. I don't think an LSTM is directly meant to be an output layer in Keras. If you want to do that, I'd bypass Keras altogether and wire up exactly what you want (and the missing pieces) in Theano.
