5
$\begingroup$

Under what conditions does the law of large numbers hold (or fail, if that is easier to describe) for independent identically distributed random variables drawn from a distribution with a finite mean and infinite or non-existent variance? Is it any different if the distribution is stationary but not independent?

$\endgroup$
1
  • 1
    $\begingroup$ For i.i.d. samples the SLLN holds if you have finite mean. No other assumption is needed. See theorem 8.21 of this book. $\endgroup$
    – Mur1lo
    Jun 27, 2016 at 2:52

1 Answer 1

4
$\begingroup$

Let $\{X_1,X_2,\ldots\}$ be a sequence of i.i.d. random variables. The (Strong) Law of Large Numbers states that:

  • If $\mathbf{E}|X_i| < +\infty$ and $\mathbf{E}X_i = \mu$

$$\frac{1}{n}\sum_{i = 1}^nX_i \overset{a.s}{\longrightarrow}\mu$$

  • If $\mathbf{E}|X_i| = +\infty$ $$\limsup\frac{1}{n}\Big|\sum_{i = 1}^nX_i\Big| =+\infty$$

Note that no assumption is made about the existence or finiteness of the variance of each $X_i$ (see chapter 6 of this book for the proofs).

Now for your question about the case of stationary sequences this paper gives a reasonably simple answer for the convergence in probability to the mean and this lecture note takes a not so simple (at least for me) take on the subject.

$\endgroup$
2
  • 1
    $\begingroup$ Dear Mur1lo-- I will accept your answer soon, but first let me try to restate it for an example & see if I have actually understood it. Suppose I have a big sample from a Pareto distribution with a tail parameter between 1 and 2. Then as the size of the sample increases, the probability that the sample mean diverges from the true mean by any particular value goes to 0, but its variance remains infinite, because there continues to be a minuscule probability of an insanely huge observation, the E.V. of which doesn't go to zero as n becomes large. Is that a correct application of your answer? $\endgroup$
    – andrewH
    Aug 18, 2016 at 1:01
  • 1
    $\begingroup$ Hi @andrewH. I believe that is the correct interpretation. I only see a problem in "because there continues to be a minuscule probability of an insanely huge observation". This statement is still true for non heavy-tailed distributions like the gaussian. The difference is that in heavy-tailed distributions those probabilities converge to zero at a lower speed, and as a consequence "the E.V. of which doesn't go to zero as n becomes large". $\endgroup$
    – Mur1lo
    Aug 18, 2016 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.