Let's say i train a RNN to predict some curve given only 1 timestep as input. So the input is x(t) and the target is x(t+1). On the next timestep the input is x(t+1) and the target is x(t+2) and so on. During training it can memorize the inputs from previous timesteps and thus make more accurate predictions and afterwards the RNN can accurately predict the curve.
What i don't understand: If i have finished training and start predicting on a random point on the curve, how can the RNN predict anything? The memory it has is from the training data and maybe we stopped training when the curve was going up, but now we are predicting somewhere where it goes down. Is the RNN able to adjust its memory during prediction? So we could let it run for say 10 timesteps by using the available data and afterwards it can start using the predictions as input again?
If the question is unclear feel free to tell me and I will try to make it clearer.
ps: I asked the same question on reddit but haven't gotten an answer so far.