Common proofs on the law of large numbers usually assume a sequence of IID random variables. If $X_1\dots X_n$ has a common expected value $\mu$, finite but not necessarily common variance (hence not necessarily identically distributed), and are uncorrelated (so not necessarily independent), can the typical proof using Chebychev's Theorem be modified in the following way to show that $\lim_{n \to \infty} P(|\bar X_n-\mu|>\epsilon) = 0$:

If $\bar X_n = \frac{1}{n}\sum X_i$, since $X_1\dots X_n$ are uncorrelated, $Var(\bar X_n)=\frac{\sum\sigma_i^2}{n^2}$ and $E(\bar X_n)=\mu$. Chebychev's inequality gives:

$$P(|\bar X_n-\mu|>\epsilon) \leq \frac{Var(\bar X_n)}{\epsilon^2} = \frac{\sum\sigma_i^2}{n^2\epsilon^2}$$

As $n \to \infty$, the right hand side would approach zero as long as $n^2$ grows faster than $\sum\sigma_i^2$ (which would be the case if all $\sigma_i^2$ are equal). Does this show the WLLN?

  • $\begingroup$ You shouldn't be able to prove this result because it isn't true: when the variances of the $X_n$ increase rapidly enough, the deviations between $\bar X_n$ and $\mu$ may become larger and larger, rather than being less than $\epsilon.$ The lack of independence produces subtler problems and I believe also prevents the result from holding even when $n^2$ grows faster than the sum of the variances. $\endgroup$ – whuber Mar 11 at 21:42
  • $\begingroup$ @whuber What if I impose that every $\sigma^2_i$ is bounded, in this case does it guarantees that the RHS of the inequality approaches zero? Is independence necessary, as I thought zero covariance between every pair of $X_i$ and $X_j$ ($i\neq j$) is sufficient per the bottom of page 11 of this link. I think this is because $Var[\bar X_n]$ is equal to the average of $\sigma^2_i$ whether $X_i$ is uncorrelated or independent? $\endgroup$ – Yandle Mar 11 at 22:17

Indeed, one may, for example, establish results of the following type:

If $\{z_i\}$ is an uncorrelated sequence with expected value $\mu<\infty$ and \begin{equation} \sum_{i=1}^{\infty}i^{-2}\sigma_i^2<\infty, \end{equation} it holds that $\bar{z}\to_p\mu$.

The result follows from Kronecker's Lemma, which says:

If some positive real sequences $\{a_i\}_1^{\infty}$ and $\{x_i\}_1^{\infty}$ satisfy that $$a_i\uparrow\infty$$ and $$\sum_{i=1}^nx_i/a_i\to C<\infty,$$ it then holds as $n\to\infty$ that $$ \frac{1}{a_n}\sum_{i=1}^nx_i\to 0 $$

The condition is clearly compatible with $\sigma_i^2\to\infty$, provided $\sigma_i^2=O(i^{1-\delta})$ for $\delta>0$. Then the terms in $\sum_{i=1}^{\infty}i^{-2}\sigma_i^2$ are $O(i^{-1-\delta})$, so that $$\sum_{i=1}^{\infty}i^{-2}\sigma_i^2<\infty.$$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.