What is the expected value of the sample variance under a linear regression with omitted variables of an AR(2) process?

Question

(I apologize for the title, it is quite a mouthful, and if someone has a better title, please feel free to edit)

Lately, I have been interested in phenomenons related to omission of variables. For example, it can be shown that the expected value of the sample variance under the inclusion of one variable $x_1$ but omission of one variable $x_2$ is $\mathbb{E}(s^2|x_1,x_2)= \sigma^2 + \frac{\sigma^2}{n-1} RSS_{x_1, \beta_2 x_2}$ where $\beta_2$ is the true coefficient of $x_2$ and $RSS_{x_1, \beta_2 x_2}$ is the residual sum of squares when running a regression with $x_1$ as predictor and $\beta_2x_2$ as outcome. Now, I am interested in getting a better expression for this when we have an $AR(2)$ process.

Let an $AR(2)$-process be given by $x_t = ax_{t-1}+bx_{t-2} + \epsilon_t$ (with the $\epsilon_i$ being independent and normal with standard deviation $\sigma$).

If we run a regression with $(x_2,\ldots, x_T)$ as the response vector and $(x_1,\ldots, x_{T-1})$ as the predicting variable, what can be said about the residual sum of squares? That is, if we (falsely) think that the process is an $AR(1)$-process, what can be said about the expected residual sum of squares?

Since I am quite unschooled in statistics (my only academic background is in mathematics), I would be interested in references (articles and books) as well as answers, even if they are only tangentially related to this question.

This is my first post here, so if something is unclear, please let me know and I'll try to make myself clearer.

Cheers!

Answer 1

Nice introductory book on the topic related to different aspects of time series models could be Introduction to Time Series and Forecasting by Brockwell and Davis among many others. Roughly speaking, the characteristic of the autoregressive process of order $p$ is linked to the partial autocorrelation function. Estimating the $AR(p)$ process: $$X_t = \sum_{j=1}^p\phi_jX_{t-j} + \varepsilon_t$$

one common solution is to apply the Durbin-Levinson (wiki on the math of Levinson recursion) method, where the residual sum of squares of $AR(p) $

$$RSS_p = \mathbb{E} \varepsilon^2= \mathbb{E}(X_t - \sum_{j=1}^p\phi_jX_{t-j})^2$$

is linked to the $RSS_{p-1}$ as:

$$RSS_p = RSS_{p-1}(1-\varphi_{pp}^2),$$

with $\varphi_{pp}$ being the partial autocorrelations or the last component of

$$\Gamma_p^{-1}\gamma_p = {([\gamma(i-j)]}_{i,j=1}^p)^{-1}[\gamma(1),\gamma(2),\dots,\gamma(p)]^\prime,$$

and $\gamma(.)$ being autocorrelation function. Thus if you apply wrong order autoregression it will cost you in theory the $(1 - \varphi_{pp}^2))$, note that in practice the estimation error also adds here. In small samples it may happen that a smaller model $(AR(1))$ is a better predictor than the true model $AR(2)$ (as the parameters has to be estimated and they are not known!). This is also known as the parsimony property of a smaller model.