I came across the below statement in this article:

"An appropriate model for analyzing this time series in a least squares framework is to take the logarithm of the data and regress it on its lagged values at lag 1 and 12. This corresponds to a multiplicative SARIMA(1, 0, 0)(1, 0, 0)12 model fitted by OLS. Another possibility would be to take first differences instead of logs which leads to very similar results, with slightly inferior fits."

Are there any general rules about when regression and ARIMA models are equivalent?

(I want to use the R package strucchange for monitoring, but I am not happy with having to fit all my models as regression models)

Thanks!

up vote 7 down vote accepted

In theory, the multiplicative SARIMA(1,0,0)(1,0,0)12 is not the same as regressing the data on its lags of order 1 and 12. To see this, the multiplicative SARIMA model is defined as:

$$ (1 - \phi_1L)(1 - \phi_2L^{12}) y_t = \varepsilon_t \,, \quad \varepsilon \sim NID(0, \sigma^2) \,, $$

where $L$ is the lag operator such that $L^i y_t = y_{t-i}$.

Expanding the product of polynomials we get:

$$ 1 - \phi_1Ly_t - \phi_2L^{12}y_t + \phi_1\phi_2L^{13}y_t = \varepsilon_t \\ y_t = \phi_1 y_{t-1} + \phi_2y_{t-12} - \phi_1\phi_2y_{t-13} + \varepsilon_t \,. $$

Thus, we can see that the multiplicative SARIMA model includes a lag of order 13 whose coefficient is restricted to be the product of coefficients $\phi_1\phi_{12}$. (For illustration, this result was used here.)

In practice, the unrestricted regression model with lags 1 and 12 may give a similar fit to the actual multiplicative SARIMA model, it will depend on the data. However, I wouldn't recommend working with ARIMA models as regression models since there are software packages that provide features and utilities specific for ARIMA models.


The strucchange package requires as input a model fitted by lm, i.e. the output of a linear regression model. My answer to this post gives an example about how to adapt the ideas implemented in strucchange to the case of ARIMA models. However, be aware that there is literature specific for the detection of structural breaks in the parameters of time series models.

Edit (added references)

Aue, A. and Horváth, L. (2012). "Structural breaks in time series". Journal of Time Series Analysis, 34(1), pp. 1-16. DOI: 10.1111/j.1467-9892.2012.00819.x.

Gombay, E. and Serban, D. (2009). "Monitoring parameter change in AR(p) time series models". Journal of Multivariate Analysis, 100(4), pp. 715-725. DOI: 10.1016/j.jmva.2008.08.005.


As the title mentions the equivalence between ARIMA and regression models, you may also be interested in this post, which compares the interpretation of an AR model with exogenous regressors and a linear regression model.

  • Thanks for your answer. Could you refer me to some of the literature on detecting structural breaks the parametres of time series? The idea is really some sort of "online" or live model validation of forecasts, using the potential break points in incoming data as a warning that the model is going off and should be re-fitted. – SiKiHe May 6 '15 at 11:49
  • 1
    @SiKiHe I have added a couple of references that I think are related to your purposes. The first one is a review, you will find more references there. – javlacalle May 6 '15 at 12:26

The quote is incorrect, misleading and generally messed up.

  1. ARIMA([1 12],0,0) - i.e. AR with lags 1 and 12 only - can be estimated by OLS as suggested by using lags 1 and 12. OLS is a very fast way to estimate AR models, when they assume normal distribution of errors. If the distribution assumption is different then MLE of AR will produce different results than stock OLS.
  2. AR(1) with 12 period multiplicative seasonality has a different lag structure than suggested by the quote, it should have at least lags 1, 12 and 13. See here the lag operator notation description.
  3. Difference models are integrated models, or ARIMA(p,1,q) where AR(p) and MA(q) are sub-components. The difference models are non-stationary. Thus, contrary to your quote, they should not be producing similar fits to AR models (even after log transformation), which are stationary. In other words, a difference model forecast would wonder away from the current point, while AR models tend to be reverting back to where the unconditional mean is.

Autoregressive models with or without other predictors can be fitted via least squares.

Of course if you have moving average processes, then you have to use non-linear optimization process which is used in ARIMA modeling.

  • 1
    Can you elaborate a bit on your answer? How do you fit them via least squares? – SiKiHe May 6 '15 at 8:14
  • 1
    @SiKiHe AR(p) models can be fitted via least squares and estimator is consistent. Lagged values of dependent variables are dealt with as usual regressors. Here is proof that LS estimator can be used when some conditions are met: jstor.org/discover/10.2307/… – Analyst May 11 '15 at 5:11

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.