How can a distribution have infinite mean and variance?

It would be appreciated if the following examples could be given:

1. A distribution with infinite mean and infinite variance.
2. A distribution with infinite mean and finite variance.
3. A distribution with finite mean and infinite variance.
4. A distribution with finite mean and finite variance.

It comes from me seeing these unfamiliar terms (infinite mean, infinite variance) used in an article I am reading, googling and reading a thread on the Wilmott forum/website, and not finding it a sufficiently clear explanation. I also haven't found any explanations in any of my own textbooks.

• case 2 in your list above is impossible. Commented Jun 10, 2015 at 15:45
• Possible duplicate of What is the difference between finite and infinite variance Commented Aug 15, 2017 at 10:45
• By asking for these four specific examples, I think this is a distinct question and should not be closed as a duplicate - although the other question is certainly relevant and helpful. Commented Aug 15, 2017 at 12:40
• Of the 4 examples only 1, 3 and 4 are actually possible and easy examples can be given for 1 and 4. Cauchy is an example of 1 and the Gaussian is an example of 4. It is impossible for the variance to be well-defined if the .mean does not exist. Hence 2 is not possible. An example of 3 would be interesting to construct. Commented Aug 15, 2017 at 14:21
• If you consider a Student $t$-distribution with different degrees of freedom $\nu$, then $t_1$ has no defined mean and an infinite second moment, so $|t_1|$ would fit (1) with infinite mean. Meanwhile $t_2$ has mean of $0$ and infinite second moment (and so variance) so fits (3). And $t_3$, indeed $t_\nu$ with $\nu>2$, has mean $0$ and finite variance $\frac{\nu}{\nu-2}$ and so fits (4) Commented Oct 28, 2021 at 16:22

The mean and variance are defined in terms of (sufficiently general) integrals. What it means for the mean or variance to be infinite is a statement about the limiting behavior for those integrals

For example, for a continuous density the mean is $$\lim_{a,b\to\infty}\int_{-a}^b x f(x)\ dx$$ (which might here be considered as a Riemann integral, say).

This can happen, for example, if the tail is "heavy enough"; either the upper or the lower part (or both) may not converge to a finite value. Consider the following examples for four cases of finite/infinite mean and variance:

1. A distribution with infinite mean and non-finite variance.

Examples: Pareto distribution with $$\alpha= 1$$, a zeta(2) distribution.

2. A distribution with infinite mean and finite variance.

Not possible.

3. A distribution with finite mean and infinite variance.

Examples: $$t_2$$ distribution. Pareto with $$\alpha=\frac{3}{2}$$.

4. A distribution with finite mean and finite variance.

Examples: Any normal. Any uniform (indeed, any bounded variable has all moments). $$t_3$$.

These notes by Charles Geyer talk about how to compute relevant integrals in simple terms. It looks like it's dealing with Riemann integrals there, which only covers the continuous case but more general definitions of integrals will cover all the cases you will be likely to require [Lebesgue integration is the form of integration used in measure theory (which underlies probability) but the point here works just fine with more basic methods]. It also covers (Sec 2.5, p13-14) why "2." isn't possible (the mean exists if the variance exists).

• +1 The reason why (2) is impossible is trivial: the variance is defined in terms of the mean. Slightly deeper is the fact that when the second moment of $X$ is finite, then the mean must be finite. For if the mean is infinite, then a fortiori the second moment must be infinite because the second moment is weighting the values of $X$ not only by the probability but also by $X$ itself ($X^2 = X\times X$). Those weights grow without bound, causing the second moment eventually to exceed the absolute value of the first moment.
– whuber
Commented Mar 27, 2014 at 14:54
• @whuber but you could define variance without reference to the mean (such as in terms of expectation of squared differences in pairs of values), so the issue is not as trivial as that. Something more like your second argument is actually needed. Commented Mar 27, 2014 at 22:11
• That's a good point, but if we accept that any alternative definition of the variance is algebraically equivalent to the usual definition for all distributions, then if it is undefined according to one definition that would logically seem to be a sufficient demonstration that it is undefined according to them all. Where alternatives like the one you mention come to the fore is in the study of stochastic processes where the various definitions are not equivalent.
– whuber
Commented Mar 28, 2014 at 15:24
• Yes, I do. A variance, being the expectation of a non-negative random variable, equals the Lebesgue integral of the positive part alone. Therefore, it is either finite or infinity (in the extended number line), no matter what. This property of being non-negative distinguishes the analysis of even moments from that of other moments, which can fail to be defined.
– whuber
Commented Nov 3, 2014 at 23:19
• The definition of variance is that it equals $\mathbb{E}[(X-\mathbb{E}(X))^2]$.
– whuber
Commented Nov 4, 2014 at 7:35

It's instructive to see what goes wrong -- the integrals are all very well, but a sample average is always finite, so what is the issue?

I'll use the Cauchy distribution, which has no finite mean. The distribution is symmetric around zero, so if it had a mean, zero would be that mean. Here are cumulative averages of two samples of ten thousand Cauchy variates (in red and black). First, the first 100, then the first 1000, then all of them. The vertical scale increases over the panels (that's part of the point)

If you had a distribution with a mean, the cumulative averages would settle down to that mean (by the law of large numbers). If you had a mean and variance, they would settle down at a known rate: the standard deviation of the $$n$$th mean would be proportional to $$1/\sqrt{n}$$.

The Cauchy averages are 'trying to' settle down to zero, but every so often you get a big value and the average gets bumped away from zero again. In a distribution with finite mean this would eventually stop happening, but with the Cauchy it never does. The averages don't go off to infinity, as they would for a non-negative variable with infinite mean, they just keep being kicked around by outliers for ever.

• Commented Jun 16, 2020 at 1:41

If a coin lands "heads" you win one cent.

If "tails", the winnings double and then if "heads" on the second toss, you win two cents.

If "tails" the second time, the winnings double again and if "heads" on the third toss, you win four cents.

And so on: $$\begin{array}{|r|c|c|c|} \hline \text{outcome} & \text{winnings} & \text{probability} & \text{product} \\ \hline \text{H} & 1 & 1/2 & 1/2 \\ \text{TH} & 2 & 1/4 & 1/2 \\ \text{TTH} & 4 & 1/8 & 1/2 \\ \text{TTTH} & 8 & 1/16 & 1/2 \\ \text{TTTTH} & 16 & 1/32 & 1/2 \\ \text{TTTTTH} & 32 & 1/64 & 1/2 \\ \vdots\quad & \vdots & \vdots & \vdots \end{array}$$ The sum of products is $$\dfrac 12 + \dfrac 12 + \dfrac 12+\cdots = +\infty,$$ so that is an infinite expected value.

That means if you pay $$\1$$ million for each coin toss, or $$\1$$ trillion, etc., then you ultimately come out ahead. How can that be, when you're unlikely to win more than a few cents each time?

The answer is that one very rare occasions, you will get a long sequence of tails, so that the winnings will compensate you for the immense expense you've incurred. That is true no matter how high the price is that you pay for each toss.

• +1 except you could better explain why the expected value is infinite. Commented Jun 22, 2020 at 11:47

Stable distributions provide nice, parametric examples of what you're looking for:

1. infinite mean and variance: $0 < \text{stability parameter} < 1$

2. N/A

3. finite mean and infinite variance: $1 \leq \text{stability parameter} < 2$

4. finite mean and variance: $\text{stability parameter} = 2$ (Gaussian)

For simplicity, suppose we are dealing with an absolutely continuous distribution with density function $$f_X$$ with some corresponding non-negative kernel function $$g_X \propto f_X$$. Suppose we consider the general $$k$$th absolute moment, which is given by the following integral expressions:

$$\mathbb{E}(|X^k|) = \int \limits_\mathbb{R} |x|^k f_X(x) \ dx = \frac{\int_\mathbb{R} |x|^k g_X(x) \ dx}{\int_\mathbb{R} g_X(x) \ dx}.$$

Broadly speaking, this integral will be finite so long as the "tails" of $$g_X$$ decrease fast enough relative to the growth of $$|x^k|$$ that their product (i.e., the integrand) yields a finite integral when taken over the whole set of real numbers. (For the specific condition required in the tails, see the analysis below.)

The norming axiom of probability theory requires that the above integral is one when $$k=0$$, and this means that the tails of the kernel function $$g_X$$ integrates to a finite positive number. This imposes a requirement on how fast the tails of the kernel function decrease to zero. However, it is possible for the tails of the kernel function $$g_X$$ to decrease fast enough to ensure that $$\int_\mathbb{R} g_X(x) \ dx$$ is finite, but not fast enough to ensure that $$\int_\mathbb{R} |x|^k g_X(x) \ dx$$ is finite for some $$k>0$$. When this happens, the above integral is infinite, and you get moments that do not exist.

Summing this up in intuitive terms, the reason you can have distributions with moments that don't exist is that probability theory imposes only weak requirements on the rate at which the tails of a distribution decrease to zero. The norming axiom imposes a weak condition that requires the tails to decrease to zero fast enough for the integral of the density to exist, but this does not impose any requirement that the integral of the density multiplied by a positive power function must exist.

Sufficient condition for finite limit: A sufficient condition for a finite integral is $$g_X(x) = \mathcal{O}(|x|^{-(k+1+\varepsilon)})$$ for some $$\varepsilon > 0$$. This condition ensures that the kernel function (and thus also the density function) decreases fast enough in its tails to yield a finite integral (see explanation of Big-O notation here). To see why this condition is sufficient, suppose the condition holds and denote the corresponding limit supremum:

$$K \equiv \limsup_{|x| \rightarrow \infty} \Bigg| \frac{f_X(x)}{|x|^{k+1+\varepsilon}} \Bigg|.$$

The condition stated here ensures that $$K < \infty$$ and we then have:

\begin{align} \mathbb{E}(|X^k|) &= \int \limits_\mathbb{R} |x|^k f_X(x) \ dx \\[6pt] &= \int \limits_{-1}^1 |x|^k f_X(x) \ dx + \int \limits_\mathbb{R} |x|^k f_X(x) \cdot \mathbb{I}(|x| \geqslant 1) \ dx \\[6pt] &\leqslant 2 + \int \limits_\mathbb{R} |x|^k f_X(x) \cdot \mathbb{I}(|x| \geqslant 1) \ dx \\[6pt] &= 2 + \int \limits_\mathbb{R} |x|^k \mathcal{O}(|x|^{-(k+1+\varepsilon)}) \cdot \mathbb{I}(|x| \geqslant 1) \ dx \\[6pt] &= 2 + \int \limits_\mathbb{R} \frac{\mathcal{O}(1)}{|x|^{1+\varepsilon}} \cdot \mathbb{I}(|x| \geqslant 1) \ dx \\[6pt] &\leqslant 2 + K \times \int \limits_\mathbb{R} \frac{1}{|x|^{1+\varepsilon}} \cdot \mathbb{I}(|x| \geqslant 1) \ dx \\[6pt] &= 2 + K \times 2 \int \limits_1^\infty \frac{1}{x^{1+\varepsilon}} \ dx \\[6pt] &= 2 + K \times 2 \Bigg[ - \frac{1}{\varepsilon x^{\varepsilon}} \Bigg]_{x=1}^{x \rightarrow \infty} \\[6pt] &= K \times 2 \Bigg[ 0 - - \frac{1}{\varepsilon} \Bigg]_{x=1}^{x \rightarrow \infty} \\[6pt] &= K \times \frac{2}{\varepsilon} < \infty, \\[6pt] \end{align}

which establishes that the integral is finite. (Note that the weaker condition $$g_X(x) = \mathcal{O}(|x|^{-(k+1)})$$ is not sufficient for this result.)

A simple example can be constructed by considering different probability measures (but they can be viewed as distributions from the same family, as in @Steve Schulist's answer) on the countably infinite sample space $$\Omega = \{1, 2, 3, \ldots\}$$. The only property used in the construction is the convergence behavior of the Riemann Zeta function $$\zeta(s) = \sum\limits_{n = 1}^\infty \frac{1}{n^s}$$. It is well known that $$\zeta(s) = \infty$$ for $$0 < s \leq 1$$ and $$\zeta(s) < \infty$$ for $$s > 1$$.

1. Infinite mean, infinite variance: $$P(X = n) = \frac{1}{n^2\zeta(2)}, n = 1, 2, \ldots$$. To be more precise, I should say infinite $$E[X^2]$$ as $$\operatorname{Var}(X)$$ is undefined when $$E[X]$$ is infinite.
2. Infinite mean and finite variance. This is impossible, as the Lyapunov's inequality implies that $$E[|X|] \leq \sqrt{E[|X|^2]}$$ -- hence if $$X$$ has finite variance, then it must have finite mean.
3. Finite mean, infinite variance: $$P(X = n) = \frac{1}{n^3\zeta(3)}, n = 1, 2, \ldots$$.
4. Finite mean, finite variance: $$P(X = n) = \frac{1}{n^4\zeta(4)}, n = 1, 2, \ldots$$.

Let $$F$$ be any distribution function for some random variable $$X,$$ continuous or not. It has infinite mean if and only if one of the integrals

$$E_F^+ = \int_0^\infty (1-F(x))\,\mathrm dx$$

and

$$E_F^- = \int_{-\infty}^0 F(x)\,\mathrm dx$$

diverges, because the mean is $$E_F = E_F^+ - E_F^-.$$ (When both integrals diverge, the mean is undefined, at least in the Lebesgue sense.)

The variance is infinite only when the distribution of $$X^2$$ has infinite mean. Because the distribution function of $$X^2$$ is given for $$t\ge 0$$ by

$$\Pr(X^2 \le t) = \Pr(-\sqrt t \le X \le \sqrt t) = F(\sqrt t) - \lim_{y\to t^+} F(-\sqrt{y}),$$

$$X$$ has infinite variance if and only if a comparable analysis holds with $$F$$ replaced by the function $$t\to F(\sqrt t).$$ This incidentally shows that infinite mean implies infinite variance (for $$t\ge 1,$$ $$1-F(\sqrt t)$$ dominates $$1-F(t),$$ so divergence of the latter implies divergence of the former), thereby eliminating case (2) of the question from the possibilities.

This gives us an easy characterization of infinite moments and a fount of examples obtained by considering, say, any function $$S$$ defined on $$[0,\infty)$$ that is right-continuous with left limits (cadlag), decreasing, and approaches $$0$$ in the limit: $$S,$$ normalized never to exceed $$1,$$ will serve for $$1-F.$$ Thus:

1. (Infinite mean and variance.) Let $$S$$ decay no more quickly than $$1/x.$$ Concrete examples: $$F(x) = 1 - 1/x^p$$ for $$x \ge 1$$ where $$0 \lt p \le 1.$$

2. (Infinite mean and finite variance.) Empty.

3. (Finite mean and infinite variance.) Let $$S$$ decay no more quickly than $$1/x^2$$ but faster than $$1/x.$$ Concrete examples: $$F(x) = 1 - 1/x^p$$ for $$x \ge 1$$ where $$1 \lt p \le 2.$$

4. (Finite mean and finite variance.) You already know lots of these.

5. (Undefined mean and infinite variance.) Let $$X$$ and $$Y$$ be positive random variables with infinite means. Choose $$0\lt \pi \lt 1$$ and take $$Z$$ to equal $$X$$ with probability $$\pi$$ and otherwise to equal $$-Y.$$ The distribution of $$Z$$ is a mixture of $$X$$ and $$-Y.$$ Moreover, all undefined-mean distributions arise this way (this is immediate from the initial analysis above). Concrete example: $$F(x) = 1 - 1/(2(x+1)^p)$$ for $$x\ge 0$$ and $$F(x) = 1/(2(1-x)^q)$$ for $$x\le 0$$ where $$0 \lt p \le 1$$ and $$0\lt q \le 1.$$

About the second distribution you are looking for, consider the random variable $$X_2 = \text{number of times you can zoom in like 10cm into a fractal}$$ then the answer is infinite with probability one, and therefore the variance is zero and the mean of the distribution has a value of infinite.

• Thats an interesting example, but for the calculations you need an extended real number system where $\infty - \infty=0$. Commented Aug 25, 2018 at 11:03

Discrete Examples for all the 3 possible situations:

1. A Distribution with infinite mean and infinite variance. mass function on Natural Numbers, $$P_X(t)=\frac{6}{\pi^2}.\frac{1}{t^2}$$ (you can check that this a valid probability mass function). For the above pmf, the mean and the variance will be, $$E(X)=\sum_{t \in \mathbb{N}}\frac{6}{\pi^2}\frac{1}{t}=\frac{6}{\pi^2}\sum_{t \in \mathbb{N}}\frac{1}{t}$$ which is a harmonic series and it will diverge. $$\therefore$$ $$Var(X)=E(X^2)-E(X)^2$$ as the mean tends to infinity, so does the variance.

2. A Distribution with finite mean and infinite variance.

Consider the following mass function on Natural Numbers

$$P_Y(s)=K\frac{1}{s^3} \text{\hspace{3mm} where s \in \mathbb{N} }$$ where $$K$$ is a positive real number such that the above mass function is valid.

From the Mean and Variance calculations, we get $$E(Y)=\sum_{s \in \mathbb{N}}K\frac{1}{s^2}=K.\frac{\pi^2}{6}\text{(from basel problem)}$$ similarly, we have $$E(Y^2)=\sum_{s \in \mathbb{N}}K\frac{1}{s}$$ which is a harmonic series and it will diverge. $$\therefore Var(Y) \text{\hspace{3mm}is infinite}$$

1. A Distribution with finite mean and finite variance. Consider the following mass function(Bernoulli Distribution) $$P_Z(0)=p \hspace{4mm} P_Z(1)=1-p$$ where $$p \in [0,1]$$ which has both finite mean and variance(very easy to check).