(Note: I am taking a first course in time series -- correct me where I am wrong.)

What happens when we fit an ARMA model to a time series when a differenced model (ARIMA with nontrivial $d$), should be used instead?

To be precise, consider the following simulation in R:

example <- arima.sim(n=2000,list(order=c(1,1,3),ar=0.9,ma=c(-1,0.1,0.01)))

When I plot the acf of this function, I get the following slowly decaying ACF, which is exactly what we expect from an ARIMA process.

enter image description here

I have been taught that this indicates the time series should be differenced. Suppose that I don't and fit an ARMA(1,1) to the data instead.

fit <- arima(example, order=c(1,0,1))

Now, if I plot the residuals, they look stationary and well-behaved. Also, here is the ACF and PACF of the residuals for "fit":

enter image description here

So the residuals look like white noise, which is what we expect from a ``good'' model.

My question is: What was the foul that I made by doing this ARMA fit when I should have differenced the series immediately? I am too inexperienced to detect a flaw in this proposed ARMA(1,1) model. (Maybe there isn't one...)


1 Answer 1


Any ARIMA($p,d,q$) model can be approximated arbitrarily closely by an ARMA($p+d, q$) model. To do this, all you need to do is to set the auto-regressive characteristic polynomial of the latter to have $d$ roots that are set very close to one (but still less than one to allow stationarity).

Details: Under an ARIMA($p,d,q$) model you have auto-regressive characteristic polynomial:

$$\phi(B) = (1-B)^d(1 - \phi_1 B - \cdots - \phi_p B^p).$$

Taking the root $r = 1 + \epsilon$ for some small value $\epsilon > 0$, you can approximate this arbitrarily closely by the atuo-regressive characteristic polynomial:

$$\phi^*(B) = \Big( 1- \frac{B}{r} \Big)^d (1 - \phi_1 B - \cdots - \phi_p B^p) = (1 - \phi_1^* B - \cdots - \phi_{p+d}^* B^{p+d}).$$

As $\epsilon \rightarrow 0$ you have $\phi^* \rightarrow \phi$, so the ARMA approximation converges to the ARIMA form.

Yours results: The above relationship means that it is unsurprising that you can get a good fit to ARIMA data using an ARMA model. In your particular example you have only used $p = 1$ values in your ARMA model, not $p+d=2$. The result is that you get a second auto-correlation line in your PACF. If you instead fit an ARMA($2,3$) model, you should get a good fit to your data, and you will probably find that the estimated auto-regressive characteristic polynomial has $d = 1$ roots that are close to unity.

You have not used set.seed in your example, so your results are not replicable. You have also fit an ARMA model with lower degree than would be required to approximate the ARIMA data ideally. This means that the above issue is conflated with use of insufficient parameters in your model. To rectify this, here is a replicable example using an ARMA($2,3$) model to approximate data from an ARIMA($1,1,3$) process:


#Generate time-series data from ARIMA(1,1,3) model
arima.model <- list(order = c(1,1,3), ar = 0.3, ma = c(-1, 0.1, 0.01));
example <- arima.sim(n = 10000, arima.model);

#Fit data to ARMA(2,3) model
arima(example, order = c(2,0,3));

arima(x = example, order = c(2, 0, 3))

         ar1     ar2     ma1      ma2      ma3  intercept
      0.0286  0.9705  0.2782  -0.8137  -0.1422    -5.3551
s.e.  0.0893  0.0892  0.0899   0.0644   0.0140     2.9047

sigma^2 estimated as 1.026:  log likelihood = -14323.05,  aic = 28660.11

As you can see from this output, the estimated auto-regressive characteristic polynomial has a root that is very close to one. The ARMA model has absorbed the differencing operation in the ARIMA data and approximated this by an additional degree in the AR part, with a root that is close to one.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.