# Ergodicity explained in layman terms

I've been told that Ergodicity gives us a practical vision of processes WSS (Wide-sense stationary) and a bunch of integrals. For me, it is not enough to fully understand it.

Could someone explain me Ergodicity in a simple way?

EDIT:

Thank you all for those interested in the question and answered, here I will share an example:

$$y(t)$$ is a random process where $$\{i(t),q(t)\}$$ are two random stationary processes, incorrelated, null mean and autocorrelation $$Ri(z) = Rq(z)$$.

$$y(t) = i(t)\cos(2\pi f_0t)−q(t)\sin(2\pi f_0t)$$

The exercise asks for mean and autocorrelation of y(t) and finally if that process is stationary or cyclostationary.

Is this process ergodic? How could I demonstrate such thing?

• See this answer of mine) on dsp.SE for a discussion about the meaning of ergodicity and an example of a process which is not stationary to any order but is wide-sense stationary, and mean-ergodic and autocovariance-ergodic. Jun 10, 2020 at 21:21

Here's the simplest way I can think of: if you watch a stochastic process long enough you're going to see every possible outcome. Not only that, but also you can obtain the probabilities of such outcomes.

What's the deal here? There are some processes where you can't have repeated trials. For instance, a coin toss is easy to replicate cross-sectionally: just get many coins, and toss them simultaneously. What about weather? Can you replicate weather on Jan 1 2018? Obviously, no. There's only one Jan 1 2018, and it will never repeat. However, if you had ergodicity you could watch weather for many days or even years, and figure what were the probabilities of different weather realization on Jan 1 2018.

Summarizing, ergodicity establishes certain equivalence between multiple trials in the same time period (cross-sectional) and prolonged observation of the same process over time (time-series). This is helpful when, particularly, cross-sectional experiment is not possible and observation over time is possible.

If you live long enough, you'll experience everything.

Robert Torricelli, https://www.brainyquote.com/quotes/robert_torricelli_404413

• Interesting answer, but while your answer is perfectly satisfactory as far as the OP in concerned, how does your answer address the OP's question which is "How do I demonstrate that this process is ergodic?" Jun 10, 2020 at 20:12
• @DilipSarwate, fair comment, but does a layman need to know how to demonstrate ergodicity? wouldn't this elevate a layman into an expert? Jun 10, 2020 at 20:37
• @DilipSarwate, I'll think of an answer though. Maybe something along the lines of "show that your process goes through all states with correct frequencies over long period of time." Jun 10, 2020 at 20:39
• I just posted my own answer to the ergodicity question. Any comments that you care to make are most welcome. Jun 14, 2020 at 20:29
• Aksakal, does "every possible outcome" mean every possible value of the process variable $y(t)$, or does it also mean every possible trajectory of values $y(t-i),y(t-i+1),\dots,y(t)$ for $i$ is some positive integer value? I suspect it's only the former, but can you enlighten? Oct 9, 2021 at 22:13

To quote Wiki with a little formatting of my own:

[...] a stochastic process is said to be ergodic if its statistical properties can be deduced from a single, sufficiently long, random sample of the process.

In other, perhaps overly simplified words, if you observe a process for a long enough time, then you can know all there is to know about the process in terms of statistical behavior, because what you are observing gets closer and closer (coverges) to the "true" (what the wiki calls ensemble) properties of the process properties.

On the same page, the counterexample with the two coins where one is fair and the other has only one outcome shows you exactly a process that is not mean-ergodic: when you pick a coin at random and start throwing it, the average of those throws will never be close to the (ensemble) mean of the process, no matter which coin you ended up picking or how many times you throw it.

From Bishop, a Markov chain is ergodic when you can run it starting from any initial distribution and end up converging to its invariant distribution (steady state, or equilibrium).

A sufficient condition for ergodicity is that you can move from any state to any other with nonzero probability in a finite number of steps.

• I wonder if it might be necessary to add "in finite time" after the nonzero probability part. Even if the chain is made of recurrent states, these states still need to be positive recurrent, no?
– Emil
May 7, 2018 at 17:07
• Your first sentence does not jibe well with the "eventually visits all parts of a sample space" meaning of "ergodic" to my intuition. The second sentence nails it though. Can you help connect the two sentences for me? Oct 9, 2021 at 22:16
• @Alexis I am not sure actually, I wrote this so long ago. I guess if there exists some initial distribution such that it doesn't converge to steady state from that distribution, then it cannot be ergodic. I can reread the chapter I referenced, or possibly develop another more rigorous answer soon. Oct 10, 2021 at 5:17

This is an answer to the question

Is this process ergodic? How could I demonstrate such thing?

regarding the random process $$\{Y(t) = I(t)\cos(2\pi f_0t)−Q(t)\sin(2\pi f_0t)\}$$ where $$\{I(t)\}$$ and $$\{Q(t)\}$$ are uncorrelated zero-mean stationary ergodic processes with identical autocorrelation function $$R_I(z) = R_Q(z) = R(z)$$.

I will take ergodicity as short-hand for mean ergodicity, that is, the requirement that for almost all sample paths $$y(t)$$ of the process $$\lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T y(t) \,\mathrm dt = E[Y(t)]=0.$$ Now, $$y(t) = i(t)\cos(2\pi f_0t)−q(t)\sin(2\pi f_0t)$$ where the sample paths $$i(t)$$ and $$q(t)$$ satisfy the ergodicity requirement, that is, \begin{align} \lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T i(t) \,\mathrm dt &= E[I(t)]=0,\\ \lim_{T\to\infty} \frac{1}{2T} \int_{-T}^T q(t) \,\mathrm dt &= E[Q(t)]=0 \end{align} but $$\frac{1}{2T} \int_{-T}^T i(t) \,\mathrm dt \approx 0,\quad \frac{1}{2T} \int_{-T}^T q(t) \,\mathrm dt \approx 0$$ does not necessarily imply that $$\frac{1}{2T} \int_{-T}^T i(t)\cos(2\pi f_0t) \,\mathrm dt \approx 0,\quad \frac{1}{2T} \int_{-T}^T q(t)\sin(2\pi f_0t) \,\mathrm dt \approx 0.$$ But with a further additional assumption about $$\{I(t)\}$$ and $$\{Q(t)\}$$ coupled with vigorous hand-waving (on my part; experts in ergodic theory might have solid reasons to see why what I am claiming is in fact completely true (or, God forbid, completely false!)), I can sort of make it work. The assumption is that $$\{I(t)\}$$ and $$\{Q(t)\}$$ are low-pass processes, that is, their common power spectral density $$S(f)$$ has value $$0$$ for all $$f > f_L$$ where $$f_L \ll f_0$$. What this says is that $$i(t)$$ and $$q(t)$$ are very slowly varying functions compared to the rapid oscillations of $$\cos(2\pi f_0t)$$ and $$\sin(2\pi f_0t)$$ and so over the short time interval $$\left[\left(k-\frac{1}{2}\right)\frac{1}{f_0},\left(k+\frac{1}{2}\right)\frac{1}{f_0}\right]$$corresponding to one period of the sinusoid of frequency $$f_0$$, $$i(t)$$ and $$q(t)$$ can be assumed to have approximately fixed value so that I can aver with a straight face that \begin{align} \int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}} i(t)\cos(2\pi f_0t) \,\mathrm dt \approx i\left(\frac{k}{f_0}\right)\int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}}\cos(2\pi f_0t) \,\mathrm dt = 0,\\ \int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}} q(t)\cos(2\pi f_0t) \,\mathrm dt \approx q\left(\frac{k}{f_0}\right)\int_{\left(k-\frac{1}{2}\right)\frac{1}{f_0}}^{\left(k+\frac{1}{2}\right)\frac{1}{f_0}}\cos(2\pi f_0t) \,\mathrm dt = 0. \end{align} Thus, by dividing the long time interval $$[-T,T]$$ into short segments of length $$\frac{1}{f_0}$$ and breaking up the integral over $$[-T,T]$$ into the sum of integrals over these short segments, we can claim that, for large $$T$$, $$\frac{1}{2T} \int_{-T}^T i(t)\cos(2\pi f_0t) \,\mathrm dt \approx 0,\quad \frac{1}{2T} \int_{-T}^T q(t)\sin(2\pi f_0t) \,\mathrm dt \approx 0,$$ hold, or at least are close enough for gummint purposes. In short,

$$\{Y(t) = I(t)\cos(2\pi f_0t)−Q(t)\sin(2\pi f_0t)\}$$ as defined above is an ergodic process if $$\{I(t)\}$$ and $$\{Q(t)\}$$ are assumed to be low-pass processes as compared to what we electrical engineers call the carrier frequency $$f_0$$.

I remark in conclusion that $$S_Y(f)$$, the power spectral density of $$\{Y(t)\}$$ is given by $$S_Y(f) = \left.\left.\frac 12\right(S(f-f_0) + S(f+f_0)\right)$$

• This wave example was added after I answered the question, so it's good to have an answer that addresses it. Although I can't call it layman explanation anymore, it is probably impossible to answer it in plain English. Jun 14, 2020 at 20:59
• If you found this answer helpful, then please consider upvoting and/or accepting it. Oct 9, 2021 at 13:58
• @kjetilbhalvorsen Since I wrote the answer above, I can't upvote it, and since I am not the OP, I can't accept the answer either. Please consider deleting your comment just above, and if you do decide to do delete your comment, please use your Moderator powers to delete this comment too, leaving only Aksakal's comment in place. Oct 9, 2021 at 19:08
• +1 for "gummint" Oct 9, 2021 at 22:14