I am training an LSTM network for time series prediction. My understanding so far is that an LSTM network is suitable for time series prediction because it keeps a 'hidden state' which gives the LSTM network a 'notion' of what has happend in the past.
So you 'feed' the network information about, say, the last 10 days (day 1-10), in order to predict the value of the 11th day. Now we want to predict for the 12th day, so we input the sequence of the last 10 days (day 2-11). However, the network still remembers what has happened on the 1st day, because of the hidden state, correct?
If resetting the hidden state between each forward pass as advised here and also standard in the Keras libary as explained here, and also explained in this pytorch tutorial, what is the use of the hidden state? Because in that case it is not 'remembered in time', as I assume that all sequences in the same batch are processed in parallel, and sample
i+1 is not aware of the hidden state produced by sample
i because they are processed in parallel. What use does the hidden state have in such a case? Because as far as I understand the hidden state in that case does not transfer information through time/between samples. Could we not just increase the sequence length to give the network knowledge about what has happened previously?
QUESTION: Assuming my understanding of the hidden state of a LSTM correct, what is the use of the hidden state if it is reset between batches?