# Theoretical corrections of the training error for time series data

With $y_1, \ldots, y_n$ a real valued time series and $\hat{f} : \mathbb{R} \to \mathbb{R}$ a (least square) estimate of the function $y \mapsto E(Y_i \mid Y_{i-1} = y)$ the training error $$\text{err} = \frac{1}{n-1} \sum_{i=2}^n (y_i - \hat{f}(y_{i-1}))^2$$ will typically underestimate the expected prediction error of $\hat{f}$ on an independent data set. Moreover, the more flexible a model we use to fit $\hat{f}$ the worse is this optimism of the training error, which renders it useless for model selection.

Various general purpose resampling or cross-validation schemes exist, also in the time series literature, to give better estimates of expected prediction error. For the quadratic loss there are several theoretical results in the regression literature on corrections of the training error, such as $C_p$-type corrections or Stein's unbiased risk estimate, see The Estimation of Prediction Error by Efron or Chapter 7 in ESL. These results are given under independence assumptions that do not hold for time series.

Can anybody help me with references to the time series literature on theoretical corrections to the training error similar to those referred to above for regression?

-
 What sort of assumptions on the error term are you making or willing to make? – cardinal Aug 17 '11 at 0:37 @cardinal, I am willing to assume stationarity and normality to begin with. The simplest example could be the AR(1) process and any references dealing with this case is of interest too. The thing is that I don't read time series literature on a regular basis, and I have not been able to find anything that is not either likelihood based (AIC and friends) or sampling based. – NRH Aug 17 '11 at 7:09