Suppose I am feeding a 2D matrix of shape (99,13) as input to a LSTM layer. I am having n number of files, where each contains (99,13) size vectors. I have decided to consider 13 as the number of features and 99 as the timesteps.
(While implementing with Keras, I have added LSTM layer as the first layer. And I have set the output_dim of the layer as 100)
But I am unable to figure out how things really within the network when we provide an input as above. I have following questions, which I am unable to answer myself.
- What kind of values do we provide as input to the LSTM cells? (xt, ht-1 used as inputs to the cell are always vectors? In my case does xt has the shape of [1,13] ? )
- When we say we have a LSTM layer as the first layer of the model, does it feeds the first input vector to all the cells in the given layer? (Eg: feed first frame of size 13 (out of 99 frames) to all n LSTM cells in the first layer? ) Likewise does it keep on feeding all the rows of the input matrix at each time step?
- What kind of a value does a LSTM cell output at each timestep? (Do we consider the cell state as the correct output from the node? Is it a single value or a vector? If it is a vector what are the dimensions? Is there a way we can guess the dimensions? I have assumed ht as a vector)
- What is meant by output_dim (output dimension) of a given layer? Does it always have to be the number of nodes in the next layer?
Please don't make this on hold or direct to any other groups. I think these questions are related to machine learning and rnn. I have read research papers, but I haven't been able to have a clear idea about how really things work inside the LSTM network.