Keras LSTM Long Term Dependencies

Question

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras.

In Keras the input data structure for X is: (nb_samples, timesteps, input_dim).

Suppose that the shape of X is: (1000, timesteps = 10, 40).

1) Does this mean that the LSTM cells will only consider ‘batches’ of 10 previous time steps ?

2) Or is the output from LSTM cells passed between these sets of 10 timesteps I.e could you capture long term dependencies 50 timesteps out?

Answer 1

It is option 1. LSTM will learn from the 10 samples.

If you like to include more history, obviously, you can increase the time step, or you can use LSTM with stateful=True. I have found stateful LSTM's tricky but here you can find more information about them.