Lets say I am using an RNN to predict future value $x_{n+1}$. This will be based on a known dataset $\{x_i \}_{i\in[1,n]}$. I assume that the expected error will be higher for $x_{n+2}$ since $x_{n+1}$ already has a non-negative expected error. So, I expect the hidden layers to slowly start producing higher errors.
Also, for systems like weather forecasting models, I expect this to be even larger. For instance, lets say I am trying to model the weather in a small region $z_j$ which doesn't have a weather station based on regions $\{z_i \}_{i\neq j}$ with weather stations (I am ignoring satellites and the like for this example). So, I train an RNN on those and use time and map data to model the weather in region $z_j$ (with the geographic features being used to make the model generalizable to $z_j$).
Has there been done any work on how errors accumulate in RNNs?
