# Ergodicity explained in layman terms

I've been told that Ergodicity gives us a practical vision of processes WSS (Wide-sense stationary) and a bunch of integrals. For me, it is not enough to fully understand it.

Could someone explain me Ergodicity in a simple way?

EDIT:

Thank you all for those interested in the question and answered, here I will share an example:

$$y(t)$$ is a random process where $$\{i(t),q(t)\}$$ are two random stationary processes, incorrelated, null mean and autocorrelation $$Ri(z) = Rq(z)$$.

$$y(t) = i(t)\cos(2\pi f_0t)−q(t)\sin(2\pi f_0t)$$

The exercise asks for mean and autocorrelation of y(t) and finally if that process is stationary or cyclostationary.

Is this process ergodic? How could I demonstrate such thing?

• See this answer of mine) on dsp.SE for a discussion about the meaning of ergodicity and an example of a process which is not stationary to any order but is wide-sense stationary, and mean-ergodic and autocovariance-ergodic. – Dilip Sarwate Jun 10 '20 at 21:21

Here's the simplest way I can think of: if you watch a stochastic process long enough you're going to see every possible outcome. Not only that, but also you can obtain the probabilities of such outcomes.

What's the deal here? There are some processes where you can't have repeated trials. For instance, a coin toss is easy to replicate cross-sectionally: just get many coins, and toss them simultaneously. What about weather? Can you replicate weather on Jan 1 2018? Obviously, no. There's only one Jan 1 2018, and it will never repeat. However, if you had ergodicity you could watch weather for many days or even years, and figure what were the probabilities of different weather realization on Jan 1 2018.

Summarizing, ergodicity establishes certain equivalence between multiple trials in the same time period (cross-sectional) and prolonged observation of the same process over time (time-series). This is helpful when, particularly, cross-sectional experiment is not possible and observation over time is possible.

If you live long enough, you'll experience everything.

Robert Torricelli, https://www.brainyquote.com/quotes/robert_torricelli_404413

• Interesting answer, but while your answer is perfectly satisfactory as far as the OP in concerned, how does your answer address the OP's question which is "How do I demonstrate that this process is ergodic?" – Dilip Sarwate Jun 10 '20 at 20:12
• @DilipSarwate, fair comment, but does a layman need to know how to demonstrate ergodicity? wouldn't this elevate a layman into an expert? – Aksakal Jun 10 '20 at 20:37
• @DilipSarwate, I'll think of an answer though. Maybe something along the lines of "show that your process goes through all states with correct frequencies over long period of time." – Aksakal Jun 10 '20 at 20:39
• I just posted my own answer to the ergodicity question. Any comments that you care to make are most welcome. – Dilip Sarwate Jun 14 '20 at 20:29

To quote Wiki with a little formatting of my own:

[...] a stochastic process is said to be ergodic if its statistical properties can be deduced from a single, sufficiently long, random sample of the process.

In other, perhaps overly simplified words, if you observe a process for a long enough time, then you can know all there is to know about the process in terms of statistical behavior, because what you are observing gets closer and closer (coverges) to the "true" (what the wiki calls ensemble) properties of the process properties.

On the same page, the counterexample with the two coins where one is fair and the other has only one outcome shows you exactly a process that is not mean-ergodic: when you pick a coin at random and start throwing it, the average of those throws will never be close to the (ensemble) mean of the process, no matter which coin you ended up picking or how many times you throw it.

From Bishop, a Markov chain is ergodic when you can run it starting from any initial distribution and end up converging to its invariant distribution (steady state, or equilibrium).

A sufficient condition for ergodicity is that you can move from any state to any other with nonzero probability in a finite number of steps.

• I wonder if it might be necessary to add "in finite time" after the nonzero probability part. Even if the chain is made of recurrent states, these states still need to be positive recurrent, no? – Emil May 7 '18 at 17:07

This is an answer to the question

Is this process ergodic? How could I demonstrate such thing?

regarding the random process $$\{Y(t) = I(t)\cos(2\pi f_0t)−Q(t)\sin(2\pi f_0t)\}$$ where $$\{I(t)\}$$ and $$\{Q(t)\}$$ are uncorrelated zero-mean stationary ergodic processes with identical autocorrelation function $$R_I(z) = R_Q(z) = R(z)$$.

I will take ergodicity as short-hand for mean ergodicity, that is, the requirement that for almost all sample paths $$y(t)$$ of the process $$\lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T y(t) \,\mathrm dt = E[Y(t)]=0.$$ Now, $$y(t) = i(t)\cos(2\pi f_0t)−q(t)\sin(2\pi f_0t)$$ where the sample paths $$i(t)$$ and $$q(t)$$ satisfy the ergodicity requirement, that is, \begin{align} \lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T i(t) \,\mathrm dt &= E[I(t)]=0,\\ \lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T q(t) \,\mathrm dt &= E[Q(t)]=0 \end{align} but $$\frac{1}{2T} \int_{-T}^T i(t) \,\mathrm dt \approx 0,\quad \frac{1}{2T} \int_{-T}^T q(t) \,\mathrm dt \approx 0$$ does not necessarily imply that $$\frac{1}{2T} \int_{-T}^T i(t)\cos(2\pi f_0t) \,\mathrm dt \approx 0,\quad \frac{1}{2T} \int_{-T}^T q(t)\sin(2\pi f_0t) \,\mathrm dt \approx 0.$$ But with a further additional assumption about $$\{I(t)\}$$ and $$\{Q(t)\}$$ coupled with vigorous hand-waving (on my part; experts in ergodic theory might have solid reasons to see why what I am claiming is in fact completely true (or, God forbid, completely false!)), I can sort of make it work. The assumption is that $$\{I(t)\}$$ and $$\{Q(t)\}$$ are low-pass processes, that is, their common power spectral density $$S(f)$$ has value $$0$$ for all $$f > f_L$$ where $$f_L \ll f_0$$. What this says is that $$i(t)$$ and $$q(t)$$ are very slowly varying functions compared to the rapid oscillations of $$\cos(2\pi f_0t)$$ and $$\sin(2\pi f_0t)$$ and so over the short time interval $$\left[\left(k-\frac{1}{2}\right)\frac{1}{f_0},\left(k+\frac{1}{2}\right)\frac{1}{f_0}\right]$$corresponding to one period of the sinusoid of frequency $$f_0$$, $$i(t)$$ and $$q(t)$$ can be assumed to have approximately fixed value so that I can aver with a straight face that \begin{align} \int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}} i(t)\cos(2\pi f_0t) \,\mathrm dt \approx i\left(\frac{k}{f_0}\right)\int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}}\cos(2\pi f_0t) \,\mathrm dt = 0,\\ \int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}} q(t)\cos(2\pi f_0t) \,\mathrm dt \approx q\left(\frac{k}{f_0}\right)\int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}}\cos(2\pi f_0t) \,\mathrm dt = 0. \end{align} Thus, by dividing the long time interval $$[-T,T]$$ into short segments of length $$\frac{1}{f_0}$$ and breaking up the integral over $$[-T,T]$$ into the sum of integrals over these short segments, we can claim that, for large $$T$$, $$\frac{1}{2T} \int_{-T}^T i(t)\cos(2\pi f_0t) \,\mathrm dt \approx 0,\quad \frac{1}{2T} \int_{-T}^T q(t)\sin(2\pi f_0t) \,\mathrm dt \approx 0,$$ hold, or at least are close enough for gummint purposes. In short,

$$\{Y(t) = I(t)\cos(2\pi f_0t)−Q(t)\sin(2\pi f_0t)\}$$ as defined above is an ergodic process if $$\{I(t)\}$$ and $$\{Q(t)\}$$ are assumed to be low-pass processes as compared to what we electrical engineers call the _carrier_frequency $$f_0$$.

I remark in conclusion that $$S_Y(f)$$, the power spectral density of $$\{Y(t)\}$$ is given by $$S_Y(f) = \left.\left.\frac 12\right(S(f-f_0) + S(f+f_0)\right)$$

• This wave example was added after I answered the question, so it's good to have an answer that addresses it. Although I can't call it layman explanation anymore, it is probably impossible to answer it in plain English. – Aksakal Jun 14 '20 at 20:59