2
$\begingroup$

This is a purely explorative question.

I asked a question here about a "central limit theorem" for random variables with infinite variance.

I did not expect it, but it turns out that even some random variables with infinite variance nevertheless converge in distribution to a "stable distribution" (although not the normal distribution).

This makes me Wonder about something which seems even more implausible: Is there a kind of "law of large numbers" for random variables with an infinite variance?

It does not necessarily have to state that the sample mean converges in probability to the expectation, but perhaps there is something that comes close?

$\endgroup$
5
$\begingroup$

Not only the (weak and strong) laws of large numbers hold for averages of iid random variables $X_i$ with no assumption on the existence of a variance, but the weak [and only the weak] law of large numbers may even occur without the existence of an expectation of $X_i$, as detailed on the corresponding Wikipedia page.

Remember that the weak law of large numbers states that there exists $\mu$ such that$$\lim_{n\to\infty}\mathbb{P}(|\bar{X}_n-\mu|>\epsilon)=0$$for every positive constant $\epsilon>0$. And that the strong law of large numbers states [informally] that $$\mathbb{P}(\lim_{n\to\infty}\bar{X}_n=\mu)=1$$In the event the expectation $\mathbb{E}[X_i]$ exists, then $\mathbb{E}[X_i]=\mu$.

Counter-examples provided in Wikipedia are

  1. $X=\sin(Z)\exp\{Z\}/Z$ when $Z\sim\mathcal{E}xp(1)$, with $\mu=\pi/2$
  2. $X=2^Z(-1)^Z/z$ when $Z\sim\mathcal{G}(1/2)$, with $\mu=-\log(2)$
  3. $X\sim F(x)$ with $$F(x)=\mathbb{I}_{x\ge e}-\frac{e\mathbb{I}_{x\ge e}}{2x\log(x)}-\frac{e\mathbb{I}_{x\le-e}}{2x\log(-x)}+\frac{\mathbb{I}_{-e\le x\le e}}{2}$$with $\mu=0$

[in the sense that the rv's have no expectation but there exits a limit $\mu$ for the sample mean].

$\endgroup$
  • $\begingroup$ +1 Beat me to it by a minute! Serves me right for writing a longer answer instead of a succinct one such as yours. $\endgroup$ – Dilip Sarwate Feb 11 '18 at 15:16
  • $\begingroup$ For future readers confused by the comments above, Xi'an's original answer was the highlighted sentence at the beginning of his answer. $\endgroup$ – Dilip Sarwate Feb 11 '18 at 15:25
  • 1
    $\begingroup$ Thank you! And sorry for not looking up the law of large numbers. If I'd done that, I'd known that the variance doesn't need to be finite. The reason I thought this to be the case is that I had only ever seen a proof that uses finite variance. This leaves me with the question: Where exactly does the sample mean in general converge to, if the expectation does not exist? the proofs you show (and on wikipedia) are not intuitive to me. $\endgroup$ – user56834 Feb 11 '18 at 18:05
  • $\begingroup$ Mathematically, in both first examples, the random variable is not integrable in the Lebesgue sense but the alternate series converges. $\endgroup$ – Xi'an Feb 11 '18 at 18:19
2
$\begingroup$

The strong law of large numbers requires only that the random variables have finite mean $\mu$ for the sample average to converge almost surely to $\mu$. There is no requirement that the variance be finite.

The weak law of large numbers also requires only that the random variables have finite mean $\mu$ but has the weaker conclusion that the sample average converges to $\mu$ in probability (instead of almost surely as with the strong law). Here too, there is no requirement that the variance be finite, though the proof is easier for the case when the variance is assumed to be finite.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.