# Wide-Sense Stationary but not ergodic

This is based on example 2.2 from Machine Learning: A Bayesian and Optimization Perspective by Theodoridis. Please note that I'm not at all familiar with ergodic theory and I'm reading this with background from Casella and Berger.

A wide-sense stationary process (WSS), by definition, is a stochastic process $(u_n)_{n \in \mathbb{Z}}$ such that for each $k$, $\mathbb{E}[u_k] = \mu$ for all $k$ (i.e., the means are identical) and if we define $$r(n, m) = \mathbb{E}[u_nu_m]$$ we have $r(n, n-k) = r(k)$ (i.e., the "autocorrelation" (what Theodoridis defines as such, technically the cross-product moment)) only depends on the difference between times.

Consider a WSS $(u_n)_{n \in \mathbb{Z}}$ for which the expected value is $\mu$ at each time, and $$\mathbb{E}[u_nu_{n-k}] = r_u(k)\text{.}$$ Let $v_n = au_n$, where $a$ is a random variable taking values in $\{0, 1\}$, each with probability $0.5$. Assuming independence of $a$ and $u_n$, we obtain $\mathbb{E}[v_n] = 0.5\mu$ and $\mathbb{E}[v_nv_{n-k}] = 0.5r_u(k)$.

Theodoridis then remarks:

Thus, $v_n$ is WSS. However, it is not covariance-ergodic. Indeed, some of the realizations will be equal to zero (when $a = 0$), and the mean value and autocorrelation, which will result from them as time averages, will be zero, which is different from the ensemble averages.

I don't understand any of the above quote, other than that $v_n$ is WSS. By "covariance-ergodic," the author talks about $$\text{cov}(k) = \lim_{N \to \infty}\dfrac{1}{2N+1}\sum_{n=-N}^{N}(u_n - \mu)(u_{n-k}-\mu)$$ with limits "in the mean-square sense" equaling $0$ (I think?).

Could someone enlighten me on this?

Let's leave aside the concept of ergodicity itself, which is rather deep, abstract and difficult, and use its consequences: when a stochastic process $\{X_i\}$ is ergodic, the "time" average (the average across the index, which often represents time, but this is not necessary) converges in probability to the ensemble average.