35
$\begingroup$

It would be appreciated if the following examples could be given:

  1. A distribution with infinite mean and infinite variance.
  2. A distribution with infinite mean and finite variance.
  3. A distribution with finite mean and infinite variance.
  4. A distribution with finite mean and finite variance.

It comes from me seeing these unfamiliar terms (infinite mean, infinite variance) used in an article I am reading, googling and reading a thread on the Wilmott forum/website, and not finding it a sufficiently clear explanation. I also haven't found any explanations in any of my own textbooks.

$\endgroup$
  • 1
    $\begingroup$ case 2 in your list above is impossible. $\endgroup$ – kjetil b halvorsen Jun 10 '15 at 15:45
  • $\begingroup$ Relevant: stats.stackexchange.com/questions/94402/… $\endgroup$ – kjetil b halvorsen Oct 19 '16 at 20:44
  • 1
    $\begingroup$ Possible duplicate of What is the difference between finite and infinite variance $\endgroup$ – kjetil b halvorsen Aug 15 '17 at 10:45
  • 2
    $\begingroup$ By asking for these four specific examples, I think this is a distinct question and should not be closed as a duplicate - although the other question is certainly relevant and helpful. $\endgroup$ – Silverfish Aug 15 '17 at 12:40
  • 1
    $\begingroup$ Of the 4 examples only 1, 3 and 4 are actually possible and easy examples can be given for 1 and 4. Cauchy is an example of 1 and the Gaussian is an example of 4. It is impossible for the variance to be well-defined if the .mean does not exist. Hence 2 is not possible. An example of 3 would be interesting to construct. $\endgroup$ – Michael Chernick Aug 15 '17 at 14:21
51
$\begingroup$

The mean and variance are defined in terms of integrals. What it means for the mean or variance to be infinite is a statement about the limiting behavior for those integrals

For example, the mean is $\lim_{a,b\to\infty}\int_{-a}^b x\ dF$ (considering this, say as a Stieltjes integral); for a continuous density this would be $\lim_{a,b\to\infty}\int_{-a}^b x f(x)\ dx$ (now as a Riemann integral, say).

This can happen, for example, if the tail is "heavy enough". Consider the following examples for four cases of finite/infinite mean and variance:

  1. A distribution with infinite mean and infinite variance.

    Examples: Pareto distribution with $\alpha= 1$, a zeta(2) distribution.

  2. A distribution with infinite mean and finite variance.

    Not possible.

  3. A distribution with finite mean and infinite variance.

    Examples: $t_2$ distribution. Pareto with $\alpha=\frac{3}{2}$.

  4. A distribution with finite mean and finite variance.

    Examples: Any normal. Any uniform (indeed, any bounded variable has all moments). $t_3$.

You can also have a distribution where the integral is undefined but doesn't necessarily pass beyond all finite bounds in the limit.


These notes by Charles Geyer talk about how to compute relevant integrals in simple terms. It looks like it's dealing with Riemann integrals there, which only covers the continuous case but more general definitions of integral (Stieltjes for example) will cover all the cases you will be likely to require [Lebesgue integration is the form of integration used in measure theory (which underlies probability) but the point here works just fine with more basic methods]. It also covers (Sec 2.5, p13-14) why "2." isn't possible (the mean exists if the variance exists).

$\endgroup$
  • 7
    $\begingroup$ +1 The reason why (2) is impossible is trivial: the variance is defined in terms of the mean. Slightly deeper is the fact that when the second moment of $X$ is finite, then the mean must be finite. For if the mean is infinite, then a fortiori the second moment must be infinite because the second moment is weighting the values of $X$ not only by the probability but also by $X$ itself ($X^2 = X\times X$). Those weights grow without bound, causing the second moment eventually to exceed the absolute value of the first moment. $\endgroup$ – whuber Mar 27 '14 at 14:54
  • 4
    $\begingroup$ @whuber but you could define variance without reference to the mean (such as in terms of expectation of squared differences in pairs of values), so the issue is not as trivial as that. Something more like your second argument is actually needed. $\endgroup$ – Glen_b Mar 27 '14 at 22:11
  • 3
    $\begingroup$ That's a good point, but if we accept that any alternative definition of the variance is algebraically equivalent to the usual definition for all distributions, then if it is undefined according to one definition that would logically seem to be a sufficient demonstration that it is undefined according to them all. Where alternatives like the one you mention come to the fore is in the study of stochastic processes where the various definitions are not equivalent. $\endgroup$ – whuber Mar 28 '14 at 15:24
  • 2
    $\begingroup$ Yes, I do. A variance, being the expectation of a non-negative random variable, equals the Lebesgue integral of the positive part alone. Therefore, it is either finite or infinity (in the extended number line), no matter what. This property of being non-negative distinguishes the analysis of even moments from that of other moments, which can fail to be defined. $\endgroup$ – whuber Nov 3 '14 at 23:19
  • 2
    $\begingroup$ The definition of variance is that it equals $\mathbb{E}[(X-\mathbb{E}(X))^2]$. $\endgroup$ – whuber Nov 4 '14 at 7:35
5
$\begingroup$

Stable distributions provide nice, parametric examples of what you're looking for:

  1. infinite mean and variance: $0 < \text{stability parameter} < 1$

  2. N/A

  3. finite mean and infinite variance: $1 \leq \text{stability parameter} < 2$

  4. finite mean and variance: $\text{stability parameter} = 2$ (Gaussian)

$\endgroup$
1
$\begingroup$

No one has mentioned the St. Petersburg paradox here; otherwise I wouldn't post in a thread this old that already has multiple answers including one "accepted" answer.

If a coin lands "heads" you win one cent.

If "tails", the winnings double and then if "heads" on the second toss, you win two cents.

If "tails" the second time, the winnings double again and if "heads" on the third toss, you win four cents.

And so on: $$ \begin{array}{|r|c|c|c|} \hline \text{outcome} & \text{winnings} & \text{probability} & \text{product} \\ \hline \text{H} & 1 & 1/2 & 1/2 \\ \text{TH} & 2 & 1/4 & 1/2 \\ \text{TTH} & 4 & 1/8 & 1/2 \\ \text{TTTH} & 8 & 1/16 & 1/2 \\ \text{TTTTH} & 16 & 1/32 & 1/2 \\ \text{TTTTTH} & 32 & 1/64 & 1/2 \\ \vdots\quad & \vdots & \vdots & \vdots \end{array} $$ The sum of products is $\dfrac 12 + \dfrac 12 + \dfrac 12+\cdots = +\infty,$ so that is an infinite expected value.

That means if you pay $\$1$ million for each coin toss, or $\$1$ trillion, etc., then you ultimately come out ahead. How can that be, when you're unlikely to win more than a few cents each time?

The answer is that one very rare occasions, you will get a long sequence of tails, so that the winnings will compensate you for the immense expense you've incurred. That is true no matter how high the price is that you pay for each toss.

$\endgroup$
-1
$\begingroup$

About the second distribution you are looking for, consider the random variable $$ X_2 = \text{number of times you can zoom in like 10cm into a fractal} $$ then the answer is infinite with probability one, and therefore the variance is zero and the mean of the distribution has a value of infinite.

$\endgroup$
  • $\begingroup$ Thats an interesting example, but for the calculations you need an extended real number system where $\infty - \infty=0$. $\endgroup$ – kjetil b halvorsen Aug 25 '18 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.