# WLLN: can expectation exist but be infinite?

Weak law of large numbers: Let $\{h_i, i = 1, \dots n\}$ be an $m \times q$ sequence of iid random variables with mean $\mu = E[h_i]$ that exists and is finite. Then $1/n \sum_{i = 1}^n h_i \rightarrow \mu$ in probability.

I don't understand why we have exists and is finite. Could you give me an example when the expectation exists but is infinite?

• St Petersburg paradox? – Dilip Sarwate Sep 30 '14 at 19:46

The expectation of a random variable $X: \{\Omega, \frak{S}, \mathbb{P}\}\to \mathbb{R}$ is the Lebesgue integral

$$\mathbb{E}[X] = \int_\Omega X(\omega)d\mathbb{P}(\omega).$$

The Lebesgue integral is constructed in a sequence of steps whereby its domain of application is broadened to encompass an ever wider variety of random variables. The first steps ultimately define the integral for variables with non-negative values: the complications of integrating functions which might oscillate arbitrarily between negative and positive values are thereby avoided. To extend the integral to variables with negative values, decompose them into their positive and negative parts:

$$X(\omega) = X^{+}(\omega) - X^{-}(\omega)$$

where $X^{+}(\omega) = X(\omega)$ when $X(\omega)\ge 0$ and $X^{+}(\omega) = 0$ otherwise; similarly, $X^{-} = (-X)^{+}$. These are readily seen to be random variables, too (that is, they will be measurable). The integral is defined to be the difference

$$\int_\Omega X(\omega)d\mathbb{P}(\omega) = \int_\Omega X^{+}(\omega)d\mathbb{P}(\omega) - \int_\Omega X^{-}(\omega)d\mathbb{P}(\omega),$$

each of which involves a non-negative random variable and therefore the meaning of its integral has already been defined.

At this point conventions may vary. The Wikipedia articles I have linked to declare that the integral is defined only when both the positive and negative integrals are finite. One could, however, allow that the integral is also defined when at most one of the integrals is finite. We could say that it equals "$+\infty$" when the integral of the positive part diverges and equals "$-\infty$" when the integral of the negative part diverges.

In this extended sense of being defined, consider a random variable $X$ with a half-Cauchy distribution. Its probability density function (PDF) $f$ is defined and equal to $0$ when $X\lt 0$ and otherwise equal to $(2/\pi)/(1+x^2)$. Thus $X^{+}=X,$ $X^{-}=0$, and by definition

$$\mathbb{E}(X) = \int_{-\infty}^{+\infty} f(x) dx = \frac{2}{\pi}\int_0^\infty \frac{x dx}{1+x^2} - \int_\mathbb{R} 0 dx.$$

Although the first integral diverges, the second obviously is finite, so we could consider this expectation to be infinite. This example answers the question, but a full appreciation requires analysis of a distribution that looks infinite but actually cannot be defined at all. The standard example is the Cauchy distribution (also known as the Student t with one degree of freedom).

For a Cauchy-distributed variable the PDF is $(1/\pi)/(1+x^2)$ everywhere. Splitting the expectation into its positive and negative parts yields

$$\mathbb{E}(X) = \frac{1}{\pi}\int_0^\infty \frac{x dx}{1+x^2} - \frac{1}{\pi}\int_{-\infty}^0 \frac{-x dx}{1+x^2}.$$

Now both sides diverge. Since an expression like "$\infty - \infty$" is nonsensical, we have no choice but to declare this expectation undefined. One way to convince yourself of this is to consider the various ways in which the integral might be calculated: they concern how the limits of $\pm \infty$ are approached. Pick any nonnegative real value $\alpha$. As a mechanism to control the relative rates at which those limits increase, define

$$f(n) = \sqrt{(1+n^2)\exp(2\pi\alpha)-1}.$$

As $n$ grows large without bound, so does $f(n)$. Therefore, if this integral indeed had a well-defined value, it would be valid to compute it as

$$\frac{1}{\pi}\int_{-\infty}^{+\infty} \frac{x dx}{1+x^2} =\,(?) \lim_{n\to\infty}\frac{1}{\pi}\int_{-n}^{f(n)} \frac{x dx}{1+x^2}$$

because both the limits, $-n$ and $f(n)$, are expanding to encompass the entire Real line. This plot of the PDF shows how $f$ is chosen to assure that the upper limit $f(n)$ extends just a little further to the right than the lower limit $-n$ extends to the left. The parts between $-n$ and $n$ balance, contributing $0$ to the expectation. The value of $f$ is chosen so that the contribution from the excess--shown in red--is always equal to $\alpha$, no matter what $n$ may be.

But a straightforward calculation gives

$$\frac{1}{\pi}\int_{-n}^{f(n)} \frac{x dx}{1+x^2} = \frac{1}{2\pi}\log(1+x^2)|_{-n}^{f(n)} = \frac{1}{2\pi}\left(\log(1+f(n)^2) - \log(1+n^2)\right)=\alpha.$$

(Using the integration endpoints $-f(n)$ and $n$ shows that $-\alpha$ is a possible value of this limit, too.) Accordingly, since this integral can be made to equal any Real number merely by varying how the limits are taken, it cannot be considered to have a definite value.

• As evidenced by the comment thread at stats.stackexchange.com/a/36028/919, the issues of finite/infinite/undefined can be consternating. Since they have come up often, I offer the present answer in an attempt to clarify the main ideas and provide definitions that seem to be consistent with how this community has tended to think of "infinite" and "undefined" expectations. – whuber Sep 30 '14 at 22:40
• I have learned in my measure theory class that a random variables is integrable if $$\int_{-\infty}^{\infty}|x|f(x)dx < \infty.$$ Isn't it the case that this definition should be used in order to claim that the expectation exist? – Kolibris Oct 2 '14 at 13:46
• The point of my reply, and why I carried out the explanation of the Lebesgue integral in such detail, is that this is a convention. If you want to have any chance of distinguishing an "undefined" from an "infinite" expectation, then you have to look more closely into what your expression means. That is what I have done. – whuber Oct 2 '14 at 14:20

There are some useful and well-known distributions which have undefined mean. I think one of the most used is Cauchy distribution. See Wikipedia article.

For example the standard Cauchy distribution has pdf $$f(x)=\frac{1}{\pi(1+x^2)}$$ So, the mean is $$E(x) = \int_{-\infty}^{\infty}x\frac{1}{\pi(1+x^2)}dx = \frac{log(1+x^2)}{2\pi}\bigg|_{-\infty}^{\infty}$$ You can see that both ends evaluates to infinity, so the expected value is undefined.

• All very true, and much better discussed in this question. More to the point, what you state is not an answer to the question asked: an example of a random variable for which the mean exists (that is, the mean is defined) but is not finite. – Dilip Sarwate Sep 30 '14 at 21:24