# Why is ergodicity not a requirement for ARIMA models besides stationarity?

I frequently read that ARIMA models must be fitted on stationary data. But stationarity does not ensure ergodicity, which I understand is necessary to deduce population parameters from a single time series sample. Why is ergodicity not a requirement for ARIMA modeling? Do we just assume it?

Also, is there an example of a ergodic, but non-stationary process? Can you forecast these types of series?

• A nonstationary process does not necessarily have a constant mean. So, if the process is ergodic (say that $\displaystyle \frac 1T \int_0^T x(t)\, \mathrm dt$ converges where $x(t)$ is some sample path to realization or "time series sample"), what exactly should it converge to? Commented Jun 5, 2017 at 20:28
• @DilipSarwate So there's no case when a process is ergodic but non-stationary? Commented Jun 6, 2017 at 18:45
• It will depend on the type of nonstationarity - suppose the process is nonstationary because its variance changes after, say, $T/2$. The process then still is ergodic for the mean if the mean stays constant over time. Commented Jun 7, 2017 at 9:46

A bit technical maybe, but stationary ARMA processes are by construction mean-ergodic (as the other answer correctly pointed out, a previous version of my answer did not spell that out clearly and wrote ergodic as mean-ergodicity is maybe the most important "flavor" of ergodicity and hence sometimes treated synonymously with erdogicity, which, as this discussion shows, it should indeed not).

First, here is a sufficient condition for mean ergodicity:

Theorem:

Let $$Y_t$$ be covariance stationary with $$E(Y_t)=\mu$$ and $$Cov(Y_t,Y_{t-j})=\gamma_j$$ such that $$\sum_{j=0}^\infty|\gamma_j|<\infty$$. Then $$\bar{Y}_T\to_p \mu$$

Proof:

We shall actually prove that $$\bar{Y}_T$$ converges to $$\mu$$ in mean square, which implies convergence in probability. Write $$\begin{eqnarray*} E(\bar{Y}_T- \mu)^2&=&E\left[(1/T)\sum_{t=1}^T(Y_t- \mu)\right]^2\\ &=&1/T^2E[\{(Y_1- \mu)+(Y_2- \mu)+\ldots+(Y_T- \mu)\}\\ &&\quad\{(Y_1- \mu)+(Y_2- \mu)+\ldots+(Y_T- \mu)\}]\\ &=&1/T^2\{[\gamma_0+\gamma_1+\ldots+\gamma_{T-1}]+[\gamma_1+\gamma_0+\gamma_1+\ldots+\gamma_{T-2}]\\ &&\quad+\ldots+[\gamma_{T-1}+\gamma_{T-2}+\ldots+\gamma_1+\gamma_0]\} \end{eqnarray*}$$ Thus, $$\begin{eqnarray*} E(\bar{Y}_T- \mu)^2&=& 1/T^2\{T\gamma_0+2(T-1)\gamma_1+2(T-2)\gamma_2+\ldots+2\gamma_{T-1}\} \end{eqnarray*}$$

Put differently, $$\begin{eqnarray*} E(\bar{Y}_T- \mu)^2&=& 1/T\{\gamma_0+2(T-1)\gamma_1/T+2(T-2)\gamma_2/T+\ldots+2\gamma_{T-1}/T\} \end{eqnarray*}$$ This expression tends to zero as $$T\to\infty$$, as $$TE(\bar{Y}_T- \mu)^2$$ remains bounded, because $$\begin{eqnarray*} TE(\bar{Y}_T- \mu)^2&=& |\gamma_0+2(T-1)\gamma_1/T+2(T-2)\gamma_2/T+\ldots+2\gamma_{T-1}/T|\\ &\leqslant&|\gamma_0|+2(T-1)|\gamma_1|/T+2(T-2)|\gamma_2|/T+\ldots+2|\gamma_{T-1}|/T\\ &\leqslant&|\gamma_0|+2|\gamma_1|+2|\gamma_2|+\ldots+2|\gamma_{T-1}|\\ &\to&c<\infty, \end{eqnarray*}$$ using summability of the autocovariances.

That is, if the autocovariances decay sufficiently quickly, mean ergodicity follows.

We next show that any causal $$ARMA(p,q)$$ process is ergodic, as it has the required summable autocovariances.

Let us look at the $$MA(\infty)$$ representation and use the triangle inequality to bound the sufficient condition for mean ergodicity of a stationary/causal process from above.

Stationarity implies that a causal, or $$MA(\infty)$$ with summable coefficients, representation of the process exists.

The claim is therefore shown if we can show that summability of the $$MA(\infty)$$ coefficients $$\sum_{j=0}^\infty|\psi_j|<\infty$$ implies $$\sum_{k=0}^\infty|\gamma_k|<\infty$$ where $$\gamma_k=\sigma^2\sum_{j=0}^{\infty}\psi_j\psi_{j+k}$$ is the $$k$$th autocovariance of an $$MA(\infty)$$-process.

We write $$\begin{eqnarray*} \sum_{k=0}^\infty|\gamma_k|&=&\sum_{k=0}^\infty\left|\sigma^2\sum_{j=0}^{\infty}\psi_j\psi_{j+k}\right|\\ &=&\sigma^2\sum_{k=0}^\infty\left|\sum_{j=0}^{\infty}\psi_j\psi_{j+k}\right|\\ &\leqslant&\sigma^2\sum_{k=0}^\infty\sum_{j=0}^{\infty}\left|\psi_j\psi_{j+k}\right|\\ &=&\sigma^2\sum_{k=0}^\infty\sum_{j=0}^{\infty}\left|\psi_j\right|\left|\psi_{j+k}\right|\\ &=&\sigma^2\sum_{j=0}^{\infty}\left|\psi_j\right|\sum_{k=0}^\infty\left|\psi_{j+k}\right|\\ &\leqslant&\sigma^2\sum_{j=0}^{\infty}\left|\psi_j\right|\sum_{k=0}^\infty\left|\psi_{k}\right|\\ &<&\infty \end{eqnarray*}$$ Here, the first inequality uses the triangle inequality. Summability of the coefficients permits interchanging the order of summation in fourth equality (and hence taking out $$|\psi_j|$$ which does not depend on $$k$$). The second inequality follows because the second summation additionally has the terms $$\psi_0,\ldots,\psi_{j-1}$$ for $$j>0$$. The last inequality then follows from summability of the coefficients.

• Is this equivalent to saying that a stationary ARMA process is a sufficient condition for ergodicity? Commented Jun 6, 2017 at 18:39
• Yes, that is the gist of it - a stationary ARMA in the sense of being casual is also ergodic. Commented Jun 6, 2017 at 18:42
• What is shown above is that ARMA processes are mean-ergodic, which is a much weaker property that ergodicity. There are examples of non-ergodic ARMA processes. Commented Aug 27, 2020 at 7:53
• @Michael, that is correct indeed, I updated my answer accordingly. Commented Aug 30, 2020 at 11:18
• @ChristophHanck, I have a couple of questions related to ergodicity: 1 and 2. You seem to be quite an expert, so perhaps you could help me with them? Commented Apr 19, 2022 at 5:19

Ergodicity and mean-ergodicity are not the same properties.

Ergodicity is a much stronger property than mean-ergodicity (mean-ergodicity just means an $$L^2$$-LLN holds). There are easy examples of ARMA processes which are not ergodic.

What was shown by previous answer is that an ARMA process is mean-ergodic. (This is simply because that $$l^1$$, the space of absolutely summable sequences, is closed under convolution, and this makes the autocovariances also $$l^1$$, which implies mean-ergodicity.)

Why is ergodicity not a requirement for ARIMA modeling?

There is no reason for it to be. These notions have different historical origins. Ergodicity was first introduced in statistical mechanics, and intended to capture the phenomenon that "time average equals ensemble average". On the other hand, ARIMA models were introduced by Box and Jenkins for time series modeling.

You can already see from the definitions that they occur in different settings. Ergodicity is a property defined for strictly stationary processes, whereas ARMA processes are considered under covariance-stationarity.

From a time series perspective, first, the strict stationarity under which ergodicity is considered is too stringent an assumption to impose on general data. Second, the weak LLN that holds for many covariance-stationary processes (e.g. under $$l^1$$-condition for the autocovariances) is empirically just as good as the strong ergodic LLN.

For a good while, these two literatures developed separately and did not talk to each other. Later there were attempts to link the two notions by characterizing when ARMA processes satisfy strong-mixing type of conditions, which is a strengthening of ergodicity for more general processes (by, e.g. Kolmogorov and co-authors). But the connection is still incomplete.

...is there an example of a ergodic, but non-stationary process?

As stated above, ergodic processes are by definition strictly stationary.