This question is basically to get a better understanding on how linear regression with ARIMA errors models are fitted. From reading online resources on this method, e.g., Prof. Rob Hyndman's online book https://otexts.com/fpp2/regarima.html, this method basically fits a linear model between the target time series and the external regressors, then models the residuals as an ARIMA process (fit an ARIMA model for it). Then both models are combined additively. However, this cannot be the case, since the linear model obtained after fitting using this method often differs from the linear model obtained using simple ordinary least squares.
As an example, in R, we could do something like
model.lm <- lm(y ~ x) model.arima <- arima(y, xreg = x, order=c(p,d,q))
and we will generally get different linear coefficients in both models (especially if
p+d+q > 0). Which leads me to suspect that the linear model from
arima() was not derived in the same way as in
lm(). Scanning briefly the source code of the ARIMA method in R, my understanding is that it will use the linear model derived using OLS (i.e., the same as in
lm()) as initial values, but then it will optimize it along with the ARIMA model parameters, but I don't understand why.
I have tried to find other online resources that discuss this in details but I came up short.
- Is it true that the linear model derived using linear regression with ARIMA errors generally will always be different from the OLS linear model?
- Is it true that the linear model derived using linear regression with ARIMA errors are initialized from the OLS, but then optimized along with the ARIMA model parameters?
- Why is it that we have to optimize the linear model as well, when we only want to model the residuals from the linear model as ARIMA model, i.e., why is that we cannot just use the model from
lm(), substract the fitted values from the target time series, and then use
xreg, then just add the resulting models (linear + ARIMA)?