Equivalence of regression models and ARIMA models?

Question

I came across the below statement in this article:

"An appropriate model for analyzing this time series in a least squares framework is to take the logarithm of the data and regress it on its lagged values at lag 1 and 12. This corresponds to a multiplicative SARIMA(1, 0, 0)(1, 0, 0)12 model fitted by OLS. Another possibility would be to take first differences instead of logs which leads to very similar results, with slightly inferior fits."

Are there any general rules about when regression and ARIMA models are equivalent?

(I want to use the R package strucchange for monitoring, but I am not happy with having to fit all my models as regression models)

Thanks!

Answer 1

In theory, the multiplicative SARIMA(1,0,0)(1,0,0)12 is not the same as regressing the data on its lags of order 1 and 12. To see this, the multiplicative SARIMA model is defined as:

$$ (1 - \phi_1L)(1 - \phi_2L^{12}) y_t = \varepsilon_t \,, \quad \varepsilon \sim NID(0, \sigma^2) \,, $$

where $L$ is the lag operator such that $L^i y_t = y_{t-i}$.

Expanding the product of polynomials we get:

$$ 1 - \phi_1Ly_t - \phi_2L^{12}y_t + \phi_1\phi_2L^{13}y_t = \varepsilon_t \\ y_t = \phi_1 y_{t-1} + \phi_2y_{t-12} - \phi_1\phi_2y_{t-13} + \varepsilon_t \,. $$

Thus, we can see that the multiplicative SARIMA model includes a lag of order 13 whose coefficient is restricted to be the product of coefficients $\phi_1\phi_{12}$. (For illustration, this result was used here.)

In practice, the unrestricted regression model with lags 1 and 12 may give a similar fit to the actual multiplicative SARIMA model, it will depend on the data. However, I wouldn't recommend working with ARIMA models as regression models since there are software packages that provide features and utilities specific for ARIMA models.

---

The strucchange package requires as input a model fitted by lm, i.e. the output of a linear regression model. My answer to this post gives an example about how to adapt the ideas implemented in strucchange to the case of ARIMA models. However, be aware that there is literature specific for the detection of structural breaks in the parameters of time series models.

Edit (added references)

Aue, A. and Horváth, L. (2012). "Structural breaks in time series". Journal of Time Series Analysis, 34(1), pp. 1-16. DOI: 10.1111/j.1467-9892.2012.00819.x.

Gombay, E. and Serban, D. (2009). "Monitoring parameter change in AR(p) time series models". Journal of Multivariate Analysis, 100(4), pp. 715-725. DOI: 10.1016/j.jmva.2008.08.005.

---

As the title mentions the equivalence between ARIMA and regression models, you may also be interested in this post, which compares the interpretation of an AR model with exogenous regressors and a linear regression model.

Answer 2

The quote is incorrect, misleading and generally messed up.

1. ARIMA([1 12],0,0) - i.e. AR with lags 1 and 12 only - can be estimated by OLS as suggested by using lags 1 and 12. OLS is a very fast way to estimate AR models, when they assume normal distribution of errors. If the distribution assumption is different then MLE of AR will produce different results than stock OLS.
2. AR(1) with 12 period multiplicative seasonality has a different lag structure than suggested by the quote, it should have at least lags 1, 12 and 13. See here the lag operator notation description.
3. Difference models are integrated models, or ARIMA(p,1,q) where AR(p) and MA(q) are sub-components. The difference models are non-stationary. Thus, contrary to your quote, they should not be producing similar fits to AR models (even after log transformation), which are stationary. In other words, a difference model forecast would wonder away from the current point, while AR models tend to be reverting back to where the unconditional mean is.

Answer 3

Autoregressive models with or without other predictors can be fitted via least squares.

Of course if you have moving average processes, then you have to use non-linear optimization process which is used in ARIMA modeling.