I seem to have stumbled on a hole in my understanding around LSTMs. In short, I cannot understand how even a simple one is actually fed samples, upon inference time/training time. Here are the details:
I have time-series weather data of temperature, measured every hour. (Not important, but just so that you have context)
During inference time, at every point in the present, I want to predict the next
5
time-steps in the future.
I know that there is a strong long-term dependency of about 24
samples in the past. (Meaning, something 24
samples in the past, is very informative to things going on now).
So my questions are:
Inference time: I have weather data coming in continuously, one sample at a time. From the present time, I want to predict 5
samples into the future. I would like to know: Literally, at every time step, what am I inputting to the LSTM? Am I inputting just the next sample, and then letting it predict 5
future samples? Am I inputting the current new sample AND say, 10
past samples, and letting it output the next 5
future samples? In either case, how does it keep track of a very long-term dependency, from 24
samples ago?
Training time: Once again, I have a time-series of weather data, say it is 1,000,000 samples long. How would I train an LSTM here, to do what I described above? I am asking, what - mechanically - am I inputting into the LSTM, at every training iteration... literally, during each training iteration, what should the input be, such that I capture strong dependencies from 24
samples ago?
Thank you!