In this post - Rob Hyndman explains that:

Even with that correction, the two models are not quite equivalent. In the Eviews code, the differencing is done before estimation, whereas in the R code the differencing is implicit in the model. In estimating the model in R, a state space representation is used and the non-stationary components are given a diffuse prior, rather than simply differenced away. (See help(arima) in R.) This will lead to different parameter estimates.

But isn't the whole point of the "I" in ARIMA, is that we difference the data to get a stationary signal, to which we can then apply AR(p) and MA(q) terms?

How can implicit differencing be achieved?

What is a diffuse prior and how does it solve the problem of non-stationary components in an ARMA/ARIMA model?

More specifically: I understand that a diffuse prior is an uninformative prior, but what does that mean in the specific context of an ARIMA or an ARMA model?


You'll want to look at State Space time series models. (Another keyword is Kalman Filter.) These models are more general than ARIMA and have models that are the equivalent of various ARIMA models.

You may find A Practical Guide to State Space Modeling helpful, as well as the Wikipedia pages on Kalman filtering.

An ARIMA model is really a formula, and an ARIMA(1,1,1) model is:

$$(1−\phi_1B)\space (1−B)y_t = c+(1+\Theta-1B)e_t$$

where the $(1−\phi_1B)$ is the AR(1) term, the $(1+\Theta-1B)e_t$ is the MA(1) term, and $(1−B)y_t$ is the first difference term. I don't know enough about it to explain in depth, but you can code this up in a State Space representation (a bunch of matrices) and get the answer. (I got the equation from Hyndman's slides, slide(s) #5.)

Note that the standard state space Local Level Model has been proven to be equivalent to an ARIMA(0,1,1) model and the standard Local Linear Trend Model is equivalent to an ARIMA(0,2,2) model. State space models can explicitly include trends and seasonal effects rather than the ARIMA approach of treating them as nuisance parameters to be eliminated.

So this is the answer to your question: The ARIMA approach seeks to eliminate trends by differencing, so that it can model AR, MA, and seasonal effects in the resulting (mostly) stationary time series. The state space approach explicitly models the trend simultaneously with the other effects and hence can also model the other effects just as the ARMA part of an ARIMA can. (In fact, it can model much more.)

I would agree with your intuition that "implicit differencing" is a confusing way to describe it (and may actually be wrong). It's actually a case of two different approaches to handling a trend: ARIMA tries to eliminate it, state space models it.

An excellent, short (but expensive) book on state space models is An Introduction to State Space Time Series Analysis by Commandeur and Koopman.

  • $\begingroup$ @CowboyTrader Yes, I'm pretty sure that a State Space model which is equivalent to an ARIMA model would have the same restrictions as the ARIMA, and ARIMA models require stationarity. But you can expand State Space models to include trend/drift, etc, whereas an ARIMA is what it is. (I don't believe that an ARIMAX model would generalize ARIMA in the same way, but I could be wrong.) $\endgroup$ – Wayne Jul 11 '19 at 13:24
  • $\begingroup$ @CowboyTrader OK. R’s implementation of ARIMA uses a State Space model under the hood, so hopefully the convergence is rapid enough. $\endgroup$ – Wayne Jul 11 '19 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.