I am doing auto-regress by usual linear regression package.
e.g. $y_t=φx+ε_t$ with $x =y_{t-1}$

My reason is that,
Auto-regression does assumes iid errors, same for linear regression. Linear Regression doesn't have assumption on independent variables. What's different is merely that the independent variables is replaced with lagged dependent variable y.

But I see AR always is fitted by specific methods, I am afraid I'm doing wrong.

  • 1
    $\begingroup$ Actually, Hamilton "Time Series Analysis" says explicitly on p. 123 that OLS is the common method of estimation of AR processes. $\endgroup$ – Richard Hardy Nov 18 '15 at 10:37

To answer the title question, fitting an AR($p$) model using OLS will yield biased estimates. The reason is that for unbiasedness, the model errors should be uncorrelated with past, current and future values of regressors, which is not the case in autoregressive models. For example, in case of AR(1)

$$ y_t=\varphi y_{t-1}+\varepsilon_t $$

(assuming zero mean for simplicity). Lag this by 1 to obtain

$$ y_{t-1}=\varphi y_{t-2}+\varepsilon_{t-1}. $$

Note that $\varepsilon_{t-1}$ enters the model of $y_{t-1}$; hence, the regressor $y_{t-1}$ will be correlated with lagged error $\varepsilon_{t-1}$. The argument is given (without proof) e.g. in this lecture note, p. 5-6.

On a positive note, OLS gives consistent estimators for an autoregressive model (see the same lecture note, p. 4-5)

Also, in my experience OLS is quite popular for fitting AR models, and is pretty standard for fitting multivariate AR, i.e. VAR, models.


If you fit by regressing $\mathbf{y}_{p+1:n}=(y_{p+1},...,y_n)^\top$ on its lags $X=[\mathbf{y}_{p:n-1},\mathbf{y}_{p-1:n-2},...,\mathbf{y}_{1:n-p}]$ the lags of that on you're going to be conditioning on the first $p$ values (for an AR(p)). If you fit by say maximum likelihood you're able to incorporate the likelihood for the first $p$ values.

If $n$ is not very large relative to $p$ is can sometimes make a substantial difference.

  • $\begingroup$ I thought that estimating by OLS would amount to effectively cutting the sample by $p$ observations so that lagged regressors could be obtained -- instead of appending the data with some arbitrary values for the negative lags. As such there would be no conditioning on the first $p$ values. $\endgroup$ – Richard Hardy Nov 17 '15 at 18:34
  • $\begingroup$ @RIchardHardy you just exactly described why it is conditioning on the first $p$ values. They're removed from $y$, but they're all still in $X$ in the regression (and in regression, $y$ is conditioned on $X$). ergo the likelihood you maximize is one conditioning on the first $p$ y-values. (You can also show it algebraically by decomposing the likelihood, it's quite straightforward. The likelihood for the AR can be split via the prediction error decomposition into two components, that for $y_{p+1},...,y_n$ and $y_1,...,y_p$, ...ctd $\endgroup$ – Glen_b -Reinstate Monica Nov 17 '15 at 21:18
  • $\begingroup$ ctd... if you write the likelihood conditional on $y_1,...,y_p$ the second term drops out and you're left with the regression model). $\endgroup$ – Glen_b -Reinstate Monica Nov 17 '15 at 21:22
  • $\begingroup$ Thanks for the explanation! Apparently, my confusion is in terminology. Also, I did not make the connection between OLS and maximum likelihood (I only thought in terms of OLS), which apparently explains conditioning. It still gives me some thought to digest. $\endgroup$ – Richard Hardy Nov 17 '15 at 21:22
  • $\begingroup$ @RIchard I was just googling for something to link you to for the algebraic version. This document in section 2 shows it for the AR(1) case; the AR(p) case works the same way. $\endgroup$ – Glen_b -Reinstate Monica Nov 17 '15 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.