In R I'm trying to fit a linear regression with autoregressive (AR(1)) errors. In David Ruppert's book it has the following code:


    MacroDiff = as.data.frame(apply(USMacroG, 2, diff))
    fit1 = arima(unemp, order=c(1,0,0), xreg=cbind(invest, government))

the above model has aic = 86.85, however when I tried my own code, the model has a different AIC value, my code is as follows:

    fit2.1 = lm( unemp ~ government + invest)
    fit2.2 = arima(fit2.1$residuals , order=c(1,0,0) , include.mean = FALSE)

for the above model aic = 102.12. As far as I can tell the two models are exactly the same, but I don't understand why the two models have very different AIC value.


There are two main differences between the models and their AIC values:

  1. The first model fits the regression and the AR(1) part simultaneously, while the second fits them stagewise. You will likely find that the fitted coefficients differ because of that.
  2. The AIC of the first model can be obtained directly by fit1$aic and it is 86.85. But how would you obtain the AIC of the second model? aic(arima(fit2$residuals , order=c(1,0,0) , include.mean = FALSE))$aic will give you 102.12, but it is not the AIC of the second model as it ignores the first stage where two regression coefficients, an intercept and error variance were estimated.
  • $\begingroup$ I don't quite understand how are the two models different from one and another. I'm trying to fit the residual of the regression model in a AR(1) process. Can you explain what you mean by the second model fits them stage-wise ? $\endgroup$ – szd116 Jun 18 '17 at 19:17
  • $\begingroup$ Consider a more extreme example of stagewise fitting: there is a full model $y=\beta_0+\beta_1 x_1+\beta_2 x_2+\varepsilon$. You can fit it as is or you can fit it stagewise, i.e. first fit $y=\gamma_0+\gamma_1 x_1+u$ and then $u=\delta_0+\delta_2 x_2+v$. You will see that in general $\hat\beta_1\neq\hat\gamma_1$, $\hat\beta_2\neq\hat\delta_2$, and $\hat\varepsilon\neq\hat v$. $\endgroup$ – Richard Hardy Jun 18 '17 at 19:39
  • $\begingroup$ I see, so these two models solve different issues. The first model is fitting the linear regression with AR(1) errors. And the second model is testing whether the residual from the linear regression fits the AR(1) process, correct ? $\endgroup$ – szd116 Jun 18 '17 at 20:13
  • $\begingroup$ Well, not testing but fitting. However, the result is different when you do it stagewise as opposed to all at once. $\endgroup$ – Richard Hardy Jun 19 '17 at 6:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.