I seem to have stumbled on a hole in my understanding around LSTMs. In short, I cannot understand how even a simple one is actually fed samples, upon inference time/training time. Here are the details:
I have time-series weather data of temperature, measured every hour. (Not important, but just so that you have context)
During inference time, at every point in the present, I want to predict the next
5time-steps in the future.
I know that there is a strong long-term dependency of about
24 samples in the past. (Meaning, something
24 samples in the past, is very informative to things going on now).
So my questions are:
Inference time: I have weather data coming in continuously, one sample at a time. From the present time, I want to predict
5 samples into the future. I would like to know: Literally, at every time step, what am I inputting to the LSTM? Am I inputting just the next sample, and then letting it predict
5 future samples? Am I inputting the current new sample AND say,
10 past samples, and letting it output the next
5 future samples? In either case, how does it keep track of a very long-term dependency, from
24 samples ago?
Training time: Once again, I have a time-series of weather data, say it is 1,000,000 samples long. How would I train an LSTM here, to do what I described above? I am asking, what - mechanically - am I inputting into the LSTM, at every training iteration... literally, during each training iteration, what should the input be, such that I capture strong dependencies from
24 samples ago?