0
$\begingroup$

This question is basically to get a better understanding on how linear regression with ARIMA errors models are fitted. From reading online resources on this method, e.g., Prof. Rob Hyndman's online book https://otexts.com/fpp2/regarima.html, this method basically fits a linear model between the target time series and the external regressors, then models the residuals as an ARIMA process (fit an ARIMA model for it). Then both models are combined additively. However, this cannot be the case, since the linear model obtained after fitting using this method often differs from the linear model obtained using simple ordinary least squares.

As an example, in R, we could do something like

model.lm <- lm(y ~ x)

model.arima <- arima(y, xreg = x, order=c(p,d,q))

and we will generally get different linear coefficients in both models (especially if p+d+q > 0). Which leads me to suspect that the linear model from arima() was not derived in the same way as in lm(). Scanning briefly the source code of the ARIMA method in R, my understanding is that it will use the linear model derived using OLS (i.e., the same as in lm()) as initial values, but then it will optimize it along with the ARIMA model parameters, but I don't understand why. I have tried to find other online resources that discuss this in details but I came up short.

Questions

  • Is it true that the linear model derived using linear regression with ARIMA errors generally will always be different from the OLS linear model?
  • Is it true that the linear model derived using linear regression with ARIMA errors are initialized from the OLS, but then optimized along with the ARIMA model parameters?
  • Why is it that we have to optimize the linear model as well, when we only want to model the residuals from the linear model as ARIMA model, i.e., why is that we cannot just use the model from lm(), substract the fitted values from the target time series, and then use arima() with no xreg, then just add the resulting models (linear + ARIMA)?
$\endgroup$
  • $\begingroup$ I am pretty sure this has been asked before, actually more than once (though yours questions are probably broader than the previous ones). You could benefit from looking up some older threads. The brief answer is, the estimation is simulataneous, so the ARMA patterns in residuals affect the regression estimates. $\endgroup$ – Richard Hardy Apr 3 at 9:42
  • $\begingroup$ Hi @RichardHardy, thanks for pointing this out. Do you any specific threads in mind? My search and SE's suggestions do not really fit my question, e.g., some ask why the linear model obtained using ARIMA(0,0,0) is not the same as OLS, but this is not really my question. $\endgroup$ – spoonboy82 Apr 3 at 10:08
  • $\begingroup$ No specific threads, unfortunately, I just remember questions like yours being raised a few times before. Sorry. $\endgroup$ – Richard Hardy Apr 3 at 10:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.