3
$\begingroup$

In time series analysis stationarity and ergodicity have different definitions and meanings:

https://en.wikipedia.org/wiki/Stationary_process

https://en.wikipedia.org/wiki/Ergodic_process

Essentially stationarity deals with the stability of an entire distribution (in a strict sense) or the first two moments (in a weak sense) given a temporal shift. However, ergodicity is need in order to give us the possibility of inferring population characteristics from just one finite sample. More precisely, ergodicity, for some moments, warrants that these sample moments converge to exact moments.

Is possible to write examples where stationarity hold but ergodicity not. In Hamilton – Time Series Analysis (1994, page 47) there is an example where the process is stationary (weakly and strictly) but not ergodic for the mean. So the sample mean is a biased estimator for the exact mean. Also from this example we can realize that ergodicity implies finite memory of the process.

However this example is given in order to underscore that this sentences (same page):

For many applications, stationarity and ergodicity turn out to amount to the same requirements.

This conflates the two concepts, rather than keeping them separate.

However, in my experience the stationarity condition is much more widely known and considered by practitioners than the ergodicity condition. In fact, several tests for stationarity are widely used, but I have never seen (a direct) test for ergodicity.

For example, in the widely used $AR(1)$ process

$$y_t = \theta_0 + \theta_1 y_{t} + \epsilon_t$$

the weak stationarity condition ($0<|\theta_1|<1$) implies ergodicity for the mean also. Is not rare to read that stationarity implies low persistency, see white noise vs random walk example. We can extend this rule on the general class of ARIMA models (see here: Why is ergodicity not a requirement for ARIMA models besides stationarity?). Therefore stationarity seems to deal with memory also.

Question: considering that ARIMA models represent the cornerstone of time series analysis, the simplification/conflation above seem me much more than an detail. Does there exist a relevant class of time series model where stationarity and ergodicity, in some form, are implied from clearly different conditions? Are there are some examples on real data? Are there graphs which could be useful for developing some intuition?

$\endgroup$
1
  • 1
    $\begingroup$ You might find this answer on dsp.SE of some interest even though it deals mostly with continuous-time processes whereas your question seems to be about discrete-time processes or time series. $\endgroup$ Aug 26 '20 at 14:19
1
$\begingroup$

Ergodicity is a property defined for strictly stationary processes, i.e. an ergodic process is by definition strictly stationary.

Note The property being shown by the answer in Why is ergodicity not a requirement for ARIMA models besides stationarity? is mean-ergodicity, which is a much weaker property than ergodicity. There are examples of ARMA processes which are not ergodic.

Every strictly stationary process $x_t$, $t = 1, 2, \cdots$, admits canonical representation $x_t(\omega) = S^t(\omega)$ for some shift transformation $S$ defined on probability space $\Omega$. A strictly stationary process $x_t$, $t = 1, 2, \cdots$, is then ergodic if $S$ has no non-trivial invariant sets (up to measure zero). See a related discussion here.

(It's a result that a strictly stationary $x_t$, $t = 1, 2, \cdots$, is ergodic if and only if a strong LLN holds for $f(x_t)$, $t = 1, 2, \cdots$, for any $f \in L^1(\Omega)$, i.e. $ \lim_{n \rightarrow \infty} \frac{1}{n} \sum_{t=1}^n f(x_t) = E[f(x_1)] $ almost surely.)

Is [it] possible to write examples where stationarity hold but ergodicity not?

This is immediate from the definition. Take two strictly stationary processes $y_t$ and $z_t$ with different distributions. Define, for some $0<p<1$, $$ x_t= \begin{cases} y_t,& \mbox{with probability } p \\ z_t,& \mbox{with probability } 1- p \end{cases}. $$ Then $x_t$ is strictly stationary but not ergodic. In fact every strictly stationary nonergodic process admits such a decomposition (just restrict $S$ to a non-trivial invariant set).

...ergodicity imply finite memory of the process...

That's incorrect. (What is true is that non-ergodicity should imply long memory for any proposed definition of "long memory".)

The phenomenon of long memory, or infinite memory, was first observed in fractional Gaussian noise (FGN), which is a strictly stationary ergodic Gaussian process. In fact, the FGN satisfies the mixing property---which is a still stronger requirement than ergodicity.

Various attempts have been made to divide ergodic processes into short ant long memory. Given the FGN example, a candidate definition of "short memory" should imply the mixing property (and everything else that's not short memory would be long memory). This motivated the introduction of strong mixing-type properties---e.g. $\alpha$-mixing, $\phi$-mixing, etc.

For many applications, stationarity and ergodicity turn out to amount to the same requirements...

Stationarity and ergodicity are trivially not the same. Colloquially one might take them to be "the same" due to background/interest of audience, limitation of data/techniques, etc.

...considering that ARIMA models represent the cornerstone of time series...Exist a relevant class of time series model where stationarity and ergodicity, in some form, are implied from clearly different condition?

(We consider ARMA models because they are useful representations---with interpretable building blocks---of a (proper) sub-family of weakly stationary time series.)

A Gaussian stationary time series if ergodic if and only if its spectral measure is absolutely continuous with respect to the Lebesgue measure (the "only if" part is easy to see). For example, a AR(1) with Gaussian innovations is ergodic---in fact it is mixing.

You may find more general characterizations of ergodic/mixing properties for sub-families of weakly stationary series in the literature. They usually involve frequency-domain conditions.

$\endgroup$
7
  • $\begingroup$ First of all thank you for your detailed answer. Only few comments. About the possibility to have stationary but not ergodic process, i know and i not posed it as question but affirmation. I cited an example about it. Regarding your affirmation that ergodicity do not imply finite memory but non ergodicity imply long memory, I m confused. With finite memory I intended also the casa in which it became negligible in some steps (as standard AR case). You intended to underscore something like this? Or what? $\endgroup$
    – markowitz
    Aug 29 '20 at 8:28
  • $\begingroup$ @markowitz "With finite memory I intended also the casa in which it became negligible in some steps (as standard AR case)"---this suggests you don't know an actual formulation/definition of infinite vs. finite memory. I would suggest you look it up. As stated in the answer, the infinite memory property was first observed for FGN---which is ergodic, in fact more than ergodic. To say an ergodic process has finite memory, according to any definition of infinite vs. finite memory in the literature, is incorrect. $\endgroup$
    – Michael
    Aug 29 '20 at 23:41
  • $\begingroup$ I know that stochastic process is a wide and complex area, I studied (and remember) only something about that. Currently I m not confident with all the concepts you used above. These are not used in the material that I used more frequently. In the comment above I only explained what i said. Its wrong? Ok. However a generic reminder to the literature is not a good answer in my view. In my opinion, efforts for clarify his explanations, keeping them simplest as possible, and referring those to the knowledge of the audience, represent the main characteristic for any goog teacher. $\endgroup$
    – markowitz
    Aug 30 '20 at 8:47
  • $\begingroup$ Underscore the behavior of ACF of FGN should be a good starting point for the point above. More in general, I m aware that several definitions of stationarity and ergodicity exist. i focused my question in ARIMA framework. In it the weak stationarity is the most used. Some detail about links between weak stationarity and wide sense ergodicity (Wikipedia definitions) should be appreciated. $\endgroup$
    – markowitz
    Aug 30 '20 at 9:02
  • $\begingroup$ Michael, since you quoted several parts of markowitz's question, I thought I would mention that have edited markowitz's interesting question for grammar. I like your answer, and it should get more votes. :) $\endgroup$
    – Alexis
    Apr 19 '21 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.