What happens when we feed a 2D matrix to a LSTM layer

Question

Suppose I am feeding a 2D matrix of shape (99,13) as input to a LSTM layer. I am having n number of files, where each contains (99,13) size vectors. I have decided to consider 13 as the number of features and 99 as the timesteps.

(While implementing with Keras, I have added LSTM layer as the first layer. And I have set the output_dim of the layer as 100)

But I am unable to figure out how things really within the network when we provide an input as above. I have following questions, which I am unable to answer myself.

1. What kind of values do we provide as input to the LSTM cells? (xt, ht-1 used as inputs to the cell are always vectors? In my case does xt has the shape of [1,13] ? )
2. When we say we have a LSTM layer as the first layer of the model, does it feeds the first input vector to all the cells in the given layer? (Eg: feed first frame of size 13 (out of 99 frames) to all n LSTM cells in the first layer? ) Likewise does it keep on feeding all the rows of the input matrix at each time step?
3. What kind of a value does a LSTM cell output at each timestep? (Do we consider the cell state as the correct output from the node? Is it a single value or a vector? If it is a vector what are the dimensions? Is there a way we can guess the dimensions? I have assumed ht as a vector)
4. What is meant by output_dim (output dimension) of a given layer? Does it always have to be the number of nodes in the next layer?

Please don't make this on hold or direct to any other groups. I think these questions are related to machine learning and rnn. I have read research papers, but I haven't been able to have a clear idea about how really things work inside the LSTM network.

Answer 1

1) X are your inputs, if you have 99 timesteps, then you have 99 vectors of size 13 each. Hence your input to each timestep is a vector that is of size 13. You will need a starting hidden state, unless you have a reason to do otherwise your beginning hidden state can be all 0's. The size of that vector is a hyperparameter you choose.

2) Keep in mind that there are not 99 LSTM cells, there is only 1 LSTM cell that is re-used 99 times for each timestep. The LSTM cell maintains a hidden state and a cell state within it that it passes forward to the next time step. But there is only 1 set of parameters being learned. Those parameters need to be able to handle all timesteps, conditional on the current input, hidden state, and cell state.

3) The cell state is not an output, however it is passed forward as an input to the next timestep. The hidden state h_t will be passed to the output as well as to the next timestep.

4) I'm not quite sure, I need a reference to the term output_dim.

This is an excellent tutorial on LSTMs: http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Answer 2

What is meant by output_dim (output dimension) of a given layer? Does it always have to be the number of nodes in the next layer?

output_dim = dimension of the LSTM hidden states.