I am building an LSTM that takes in time-series financial data. My dataset is made up of IDs (each ID is a certain stock), and timestamps. For each ID at each timestamp, there are a number of features and a label.
My LSTM takes in as input batches of shape (50, 25, 108). The 50 corresponds to 50 different IDs, 25 corresponds to 25 timestamps for each of these IDs, all over the same parallel 25 points in time and the 108 corresponds to the 108 features I have for each ID at each timestamp which can influence its price. Each timestamp also has a label, y. For each of these 50 sequences of 25 timestamps, my LSTM is is trying to predict the value of y at this 25th timestamp - the last label of the sequence.
So, in short, my network takes in 50 different sequences across the same 25 points in time, and so the RNN is unrolled over 25 time steps and uses the output of the last as its prediction.
For the sake of simplicity, say my training dataset consists of 9 IDs and 25 timestamps and instead of my LSTM taking input of shape [50, 25, 108] it is instead [3, 5, 108]. Therefore, I would batch this data and pass it through my LSTM as 3 IDs over 5 timestamps (i.e each batch would be 3 sequences spanning the same 5 timestamps parallel in time). So the first batch would be IDs 0, 1 and 2, each over timestamps 0, 1, 2, 3, 4. The next batch would be the same IDs over timestamps 5, 6, 7, 8, 9. I would pass through 5 of these batches so that all 25 timestamps for these 3 IDs have gone through the network. It is at this point that I would reset the state of the LSTM, and then pass in 5 batches across IDs 3, 4, and 5.
Is this the correct thinking of when to reset the LSTM's state - at the beginning of each time series? My thinking is that for each batch along the 25 timestamps for a given ID, the LSTM's memory is valuable as these timestamps proceed the timestamps of the previous batch. However, when I then move to timestamp 0 of a new set of IDs, as this is the first timestamp in my data, the LSTM cannot use its memory from previous points in time to influence its predictions, and so the state of the LSTM is reset.
Secondly, once my LSTM has been trained and I am now testing its accuracy on unseen data without tuning and optimising its parameters, do I still use the same logic behind resetting the state? Do I continue to reset it each time I pass in the first timestamp of an ID? Or should resetting only be performed when training?