How can differencing in an ARIMA model be implicit instead of explicit?

In this post - Rob Hyndman explains that:

Even with that correction, the two models are not quite equivalent. In the Eviews code, the differencing is done before estimation, whereas in the R code the differencing is implicit in the model. In estimating the model in R, a state space representation is used and the non-stationary components are given a diffuse prior, rather than simply differenced away. (See help(arima) in R.) This will lead to different parameter estimates.

But isn't the whole point of the "I" in ARIMA, is that we difference the data to get a stationary signal, to which we can then apply AR(p) and MA(q) terms?

How can implicit differencing be achieved?

What is a diffuse prior and how does it solve the problem of non-stationary components in an ARMA/ARIMA model?

More specifically: I understand that a diffuse prior is an uninformative prior, but what does that mean in the specific context of an ARIMA or an ARMA model?

1 Answer

You'll want to look at State Space time series models. (Another keyword is Kalman Filter.) These models are more general than ARIMA and have models that are the equivalent of various ARIMA models.

You may find A Practical Guide to State Space Modeling helpful, as well as the Wikipedia pages on Kalman filtering.

An ARIMA model is really a formula, and an ARIMA(1,1,1) model is:

$$(1−\phi_1B)\space (1−B)y_t = c+(1+\Theta-1B)e_t$$

where the $(1−\phi_1B)$ is the AR(1) term, the $(1+\Theta-1B)e_t$ is the MA(1) term, and $(1−B)y_t$ is the first difference term. I don't know enough about it to explain in depth, but you can code this up in a State Space representation (a bunch of matrices) and get the answer. (I got the equation from Hyndman's slides, slide(s) #5.)

Note that the standard state space Local Level Model has been proven to be equivalent to an ARIMA(0,1,1) model and the standard Local Linear Trend Model is equivalent to an ARIMA(0,2,2) model. State space models can explicitly include trends and seasonal effects rather than the ARIMA approach of treating them as nuisance parameters to be eliminated.

So this is the answer to your question: The ARIMA approach seeks to eliminate trends by differencing, so that it can model AR, MA, and seasonal effects in the resulting (mostly) stationary time series. The state space approach explicitly models the trend simultaneously with the other effects and hence can also model the other effects just as the ARMA part of an ARIMA can. (In fact, it can model much more.)

I would agree with your intuition that "implicit differencing" is a confusing way to describe it (and may actually be wrong). It's actually a case of two different approaches to handling a trend: ARIMA tries to eliminate it, state space models it.

An excellent, short (but expensive) book on state space models is An Introduction to State Space Time Series Analysis by Commandeur and Koopman.