Are all ergodic random processes (at least wide sense) stationary?

Question

If not, please provide a simple example of a non-stationary process that is ergodic (in mean and covariance).

Answer 1

I answered a similar question over on dsp.SE. For the convenience of those too lazy to mosey on over there for a look, or too proud to stoop to reading what mere engineers might be writing, and also for the sake of completeness of stats.SE as a repository of questions and answers on statistics and related subjects, I am reproducing below a part of my answer on dsp.SE.

Can there be a random process that is not stationary but is ergodic? Well, NO, not if by ergodic we mean ergodic in every possible way one can think of: for example, if we measure the fraction of time during which a long segment of the sample path $x(t)$ has value at most $\alpha$, this is a good estimate of $P(X(t) \leq \alpha) = F_X(\alpha)$, the value of the (common) CDF $F_X$ of the $X(t)$'s at $\alpha$ if the process is assumed to be ergodic with respect to the distribution functions. But, we can have random processes that are not stationary but are nonetheless mean-ergodic and autocovariance-ergodic. For example, consider the process $\{X(t)\colon X(t)= \cos (t + \Theta), -\infty < t < \infty\}$ where $\Theta$ takes on four equally likely values $0, \pi/2, \pi$, and $3\pi/2$. Note that each $X(t)$ is a discrete random variable that, in general, takes on four equally likely values $\cos(t), \cos(t+\pi/2)=-\sin(t), \cos(t+\pi) = -\cos(t)$ and $\cos(t+3\pi/2)=\sin(t)$, It is easy to see that in general $X(t)$ and $X(s)$ have different distributions, and so the process is not even first-order stationary. On the other hand, $$E[X(t)] = \frac 14\cos(t)+ \frac 14(-\sin(t)) + \frac 14(-\cos(t))+\frac 14 \sin(t) = 0$$ for every $t$ while \begin{align} E[X(t)X(s)]&= \left.\left.\frac 14\right[\cos(t)\cos(s) + (-\cos(t))(-\cos(s)) + \sin(t)\sin(s) + (-\sin(t))(-\sin(s))\right]\\ &= \left.\left.\frac 12\right[\cos(t)\cos(s) + \sin(t)\sin(s)\right]\\ &= \frac 12 \cos(t-s). \end{align} In short, the process has zero mean and its autocorrelation (and autocovariance) function depends only on the time difference $t-s$, and so the process is wide sense stationary. But it is not first-order stationary and so cannot be stationary to higher orders either. Now, when the experiment is performed and the value of $\Theta$ is known, we get the sample function which clearly must be one of $\pm \cos(t)$ and $\pm \sin(t)$ which have DC value $0$ which equals $E[X(t)] 0$, and whose autocorrelation function is $\frac 12 \cos(\tau)$, same as $R_X(\tau)$, and so this process is mean-ergodic and autocorrelation-ergodic (autocovariance-ergodic too) even though it is not stationary at all. In closing, I remark that the process is not ergodic with respect to the distribution function, that is, it cannot be said to be ergodic in all respects.