# Law of Large Numbers for Covariance Stationary Processes… Difference and Relationship between LLN and Ergodicity

We have a covariance stationary time series. We must assume that the time series was produced by an ergodic process if we are to make the bridge between the realization of the time series that we observe and the population that might have generated it. The assumption of an ergodic process amounts to assuming that the mean of our time series converges in probability to the population mean.

The Law of Large Numbers for a covariance stationary process doesn't seem to add anything to the assumption of ergodicity we've made. Or does it?

(1) What does the Law of Large Numbers for a covariance stationary process give us that we did not already know, having assumed ergodicity? (2) Ergodicity is something untestable about the process that created the time series. The process is either ergodic or not, and we can't know it. Can we think of the LLN for a process as a statement about conditions under which LLN holds, hence a statement about conditions when we can expect a process to be ergodic?

• ColorStatistics: I'm pretty certain that egrodicity is needed any time you want to do any type of statistical inference on one observed time series. ( it's a stronger assumption than LLN because it says that time averages of observations over some horizon converge to averages at some time tstar if you calculate the average at tstar when the series was generated over and over again ). Halmos has a book on ergodicity but I've never read it. Hopefully someone can say more regarding your question. – mlofton Dec 4 '18 at 15:45
• @mlofton: Thank you for comment. I do not understand why ergodicity is an assumption that tells us more about the convergence of the time mean to the population mean, than does the LLN. I suspect that a key reason why I do not understand the difference between the two is that I do not understand the difference between the convergence in probability, assumed in ergodicity, and m.s. (mean square convergence), concluded as a result of LLN. Hoping someone can add some clarity as to these differences. Source for statements on converge in ergodiciyy vs. LLN: J.Hamilton, "Time Series Analysis". – ColorStatistics Dec 4 '18 at 22:40
• Like I said, I don't have a good understanding of ergodicity so hopefully someone else can comment because I don't want to lead you in the wrong direction. ergodicity is definitely a time-series concept that is quite deep and usually just assumed to be true. – mlofton Dec 5 '18 at 16:09

I'm dealing right now with this kind of topic for my last physics master degree exam, so I'll try to give what I think is the best way to tackle the question. Ergodicity in general is, as you stated, an intrinsic property of a dynamical system. Specifically, this coincides with the statement $$\lim_{T \to \infty}\frac{1}{T}\int_{0}^{T} dt f(x(t)) = \langle f(x) \rangle$$, so that given a certain well-behaved function $$f(x)$$ (observable) its time average has the distribution one as a almost certain limit. This happens $$\textit{if and only if}$$ the only invariant subsets of our dynamical space $$\Omega$$ are the $$\emptyset$$ one and the whole space itself. This is a very strong condition, which certainly implies LLN, which is evident taking as observable $$f(x)$$ the random variable $$X$$ itself. However, we could talk about ergodic properties of a system without meaning ergodicity in general, which is usually really hard to prove. In this framework the LLN on its own provides an $$\textit{ergodic property}$$ for the mean only! Other ergodic properties can in principle be studied, for instance related to the correlation function or to the distribution, meaning that the time correlation in the first case, and the normalized histogram in the second case, tend both to their distribution value. These various ergodic properties are actually more readily demonstrable than ergodicity in general. What I think you're missing is the definition of ergodicity, which is not related to mean ergodicity only, but to every function of the random variable $$X$$. I hope that this explanation is clear enough. For any doubt about ergodicity you can check about Birkhoff's theorem.