# What makes the mean of some distributions undefined?

Many PDFs range from minus to positive infinity, yet some means are defined and some are not. What common trait makes some computable?

• Convergent integrals.
– Sycorax
Commented Sep 2, 2016 at 3:08
• @Cagdas That remark does not appear to be correct. There are plenty of heavy-tailed processes. Their divergent expectations are manifest as extreme variability in long-run averages. For a convincing application of a Cauchy model, for instance, see Douglas Zare's post at stats.stackexchange.com/a/36037/919.
– whuber
Commented Sep 2, 2016 at 13:29
• @CagdasOzgenc "There is no such physical process to generate such data in real life." - as a physicist, I'm offended and demand immediate apology on behalf or Mr Breit and Herr Wigner. Commented Sep 2, 2016 at 18:39
• @CagdasOzgenc: You should read Black Swan by Taleb to see just how wrong that reasoning is. While heuristically there might not be a process which perfectly generates a distribution with an undefined mean or infinite mean, there are plenty of examples where people underestimate just how fat the tails are of their distribution and proceed to calculate means, whereas the true distribution has a mean that is completely different and usually right-skewed. This kind of improper reasoning led to many risk-assessment gafs in finance where risk is underestimated by many orders of magnitude. Commented Sep 2, 2016 at 19:10
• @CagdasOzgenc, you realize your last comment makes no sense? Commented Sep 4, 2016 at 3:19

The mean of a distribution is defined in terms of an integral (I'll write it as if for a continuous distribution - as a Riemann integral, say - but the issue applies more generally; we can proceed to Stieltjes or Lebesgue integration to deal with these properly and all at once):

$$E(X) = \int_{-\infty}^\infty x f(x)\, dx$$

But what does that mean? It's effectively a shorthand for

$$\stackrel{\lim}{_{a\to\infty,b\to\infty}} \int_{-a}^b x\, f(x)\, dx$$

or

$$\stackrel{\lim}{_{a\to\infty}} \int_{-a}^0 x f(x)\, dx \, +\, \stackrel{\lim}{_{b\to\infty}} \int_{0}^b x f(x)\, dx$$

(though you could break it anywhere not just at 0)

The problem comes when the limits of those integrals are not finite.

So for example, consider the standard Cauchy density, which is proportional to $\frac{1}{1+x^2}$ ... note that

$$\stackrel{\lim}{_{b\to\infty}} \int_{0}^b \frac{x}{1+x^2}\, dx$$

let $u=1+x^2$, so $du=2x\,dx$

$$=\,\stackrel{\lim}{_{b\to\infty}}\frac12 \int_{1}^{1+b^2} \frac{1}{u}\, du$$

$$=\,\stackrel{\lim}{_{b\to\infty}} \frac{_1}{^2}\ln(u)\Bigg |_{1}^{1+b^2}$$

$$=\,\stackrel{\lim}{_{b\to\infty}} \frac{_1}{^2}\ln(1+b^2)$$

which isn't finite. The limit in the lower half is also not finite; the expectation is thereby undefined.

Or if we had as our random variable the absolute value of a standard Cauchy, its entire expectation would be proportional to that limit we just looked at (i.e. $\stackrel{\lim}{_{b\to\infty}} \frac12\ln(1+b^2)$).

On the other hand, some other densities do continue out "to infinity" but their integral does have a limit.

• You can (of course) also see the same thing in similar discrete probability distributions. Take a distribution where the probability if $n$ occurring, for integer $n>0$, is proportional to $\frac{1}{n^2}$. The sum of probabilities is finite (which is just as well since it needs to have limit 1: actually our constant must be $\frac{6}{\pi^2}$ or whatever it is), but since the sum of $\frac{1}{n}$ diverges it has no mean. Whereas if we choose a probability proportional to $\frac{1}{n^3}$ then the mean involves a sum of $\frac{1}{n^2}$ and we're fine, that's "small enough" that it converges. Commented Sep 2, 2016 at 11:27
• Yes, $\frac{6}{\pi^2}$ is the scaling constant for that (to make it sum to1). Commented Sep 2, 2016 at 11:45
• Are you implying that any compact-supported distribution may have a mean defined?
– dodo
Commented Apr 13, 2023 at 11:32
• You can define a mean when the support is non-compact (e.g. see the Gaussian). If you mean to ask "did I claim in my answer that you can always do so when the support is compact" I don't think that claim is anywhere there, but if you want to ask in any detail about whether that's the case or if there are counterexamples, best to post a new question. (Certainly if the support is compact and $f$ is bounded, the convergence of the integral is guaranteed and the mean is defined, though that's a stronger constraint than what you asked about). Commented Apr 14, 2023 at 0:19
• Loosely if the variable has support on $[a,b]$ then $\int_a^b x.f(x) \,dx \leq b \int_a^b f(x) \, dx = b$. Can you find a way to break that working properly? Commented Apr 14, 2023 at 0:26

The other answers are good, but might not convince everyone, especially people who take one look at the Cauchy distribution (with $x_0 = 0$) and say it's still intuitively obvious that the mean should be zero.

The reason the intuitive answer is not correct from the mathematical perspective is due to the Riemann rearrangement theorem (video).

Effectively what you're doing when you're looking at a Cauchy and saying that the mean "should be zero" is that you're splitting down the "center" at zero, and then claiming the moments of the two sizes balance. Or in other words, you're implicitly doing an infinite sum with "half" the terms positive (the moments at each point to the right) and "half" the terms negative (the moments at each point to the left) and claiming it sums to zero. (For the technically minded: $\int_{0}^\infty f(x_0+r)r\, dr - \int_{0}^{\infty} f(x_0-r)r\, dr = 0$)

The Riemann rearrangement theorem says that this type of infinite sum (one with both positive and negative terms) is only consistent if the two series (positive terms only and negative terms only) are each convergent when taken independently. If both sides (positive and negative) are divergent on their own, then you can come up with an order of summation of the terms such that it sums to any number. (Video above, starting at 6:50)

So, yes, if you do the summation in a balanced manner from 0 out, the first moments from the Cauchy distribution cancel out. However, the (standard) definition of mean doesn't enforce this order of summation. You should be able to sum the moments in any order and have it be equally valid. Therefore, the mean of the Cauchy distribution is undefined - by judiciously choosing how you sum the moments, you can make them "balance" (or not) at practically any point.

So to make the mean of a distribution defined, the two moment integrals need to each be independently convergent (finite) around the proposed mean (which, when you do the math, is really just another way of saying that the full integral ($\int_{-\infty}^\infty f(x)x\, dx$) needs to be convergent). If the tails are "fat" enough to make the moment for one side infinite, you're done. You can't balance it out with an infinite moment on the other side.

I should mention that the "counter intuitive" behavior of things like the Cauchy distribution is entirely due to problems when thinking about infinity. Take the Cauchy distribution and chop off the tails - even arbitrarily far out, like at plus/minus the xkcd number - and (once re-normalized) you suddenly get something that's well behaved and has a defined mean. It's not the fat tails in-and-of-themselves that are an issue, it's how those tails behave as you approach infinity.

• Nice. I wonder if its possible to give an exlicit "order of summation" that leads to, say, two. Commented Sep 2, 2016 at 20:56
• @MatthewDrury: p_i and n_i denote positive and negative numbers. Successively find p_i and n_i so that integral over [n_i , p_i] is 2+(1/i) and integral over [n_{i+1},p_i] is 2-(1/i). One could do this explicitly using R, matlab or mathematica, but only for a finite number of terms. Commented Sep 8, 2016 at 10:06

General Abrial and Glen_b had perfect answers. I just want to add a small demo to show you the mean of Cauchy distribution does not exist / does not converge.

In following experiment, you will see, even you get a large sample and calcluate the empirical mean from the sample, the numbers are quite different from experiment to experiment.

set.seed(0)
par(mfrow=c(1,2))
experiments=rep(1e5,100)
mean_list_cauchy=sapply(experiments, function(n) mean(rcauchy(n)))
mean_list_normal=sapply(experiments, function(n) mean(rnorm(n)))
plot(mean_list_cauchy,ylim=c(-10,10))
plot(mean_list_normal,ylim=c(-10,10))


You can observe that we have $100$ experiments, and in each experiment, we sample $1\times 10^5$ points from two distributions, with such a big sample size, the empirical mean across different experiments should be fairly close to true mean. The results shows Cauchy distribution does not have a converging mean, but normal distribution has.

EDIT:

As @mark999 mentioned in the chat, we should argue the two distributions used in the experiment has similar "variance" (the reason I use quote is because Cauchy distribution variance is also undefined.). Here is the justification: their PDF are similar.

Note that, by looking at the PDF of Cauchy distribution, we would guess it is $0$, but from the experiments we can see, it does not exist. That is the point of the demo.

curve(dnorm, -8,8)
curve(dcauchy, -8,8)


• I don't think this shows that the Cauchy distribution has no mean. You could get similar results if you replaced the Cauchy distribution by a normal distribution with a suitably large variance. Commented Sep 2, 2016 at 11:34
• good point @mark999, I will edit my answer to address this problem. Commented Sep 2, 2016 at 11:41
• Is it possible to figure out from PDF of Cauchy distribution that it has no mean, probably by looking at it's fat tails ? Commented Sep 2, 2016 at 13:18
• Perhaps you had something like this in mind? stats.stackexchange.com/questions/90531/…
– Sycorax
Commented Sep 2, 2016 at 15:16

The Cauchy distribution is a disguised form of a very fundamental distribution, namely the uniform distribution on a circle. In formulas, the infinitesimal probability is $d\theta/2\pi$, where $\theta$ is the angle coordinate. The probability (or measure) of an arc $A\subset \mathbb S^1$ is $\mathtt{length}(A)/2\pi$. This is different from the uniform distribution $U(-\pi,\pi)$, though measures are indeed the same for arcs not containing $\pi$. For example, on the arc from $\pi-\varepsilon$ counter-clockwise to $-\pi+\varepsilon\ (=\pi+\varepsilon \mod 2\pi)$, the mean of the distribution on the circle is $\pi$. But the mean of the uniform distribution $U(-\pi,\pi)$ on the corresponding union of two disjoint intervals, each of length $\varepsilon/2\pi$, is zero.

Since the distribution on the circle is rotationally symmetric, there cannot be a mean, median or mode on the circle. Similarly, higher moments, such as variance, cannot make sense. This distribution arises naturally in many contexts. For example, my current project involves microscope images of cancerous tissue. The very numerous objects in the image are not symmetric and a "direction" can be assigned to each. The obvious null hypothesis is that these directions are uniformly distributed.

To disguise the simplicity, let $\mathbb S^1$ be the standard unit circle, and let $p=(0,1)\in\mathbb S^1$. We define $x$ as a function of $\theta$ by stereographical projection of the circle from $p$ onto the $x$-axis. The formula is $x=\tan(\theta/2)$. Differentiating, we find $d\theta/2 = dx/(1+x^2)$. The infinitesimal probability is therefore $\frac{d\theta}{\pi(1+x^2)}$, the usual form of the Cauchy distribution, and "Hey, presto!", simplicity becomes a headache, requiring treatment by the subtleties of integration theory.

In $\mathbb S^1 \setminus \{p\}$, we can ignore the absence of $p$ (in other words, reinstate $p\in\mathbb S^1$) for any consideration such as mean or higher order moment, because the probability of $p$ (its measure) is zero. So therefore the non-existence of mean and of higher moments carries over to the real line. However, there is now a special point, namely $-p = (0,-1)$, which maps to $0\in\mathbb R$ under stereographic projection and this becomes the median and mode of the Cauchy distribution.

• The Cauchy distribution has a median and mode.
Commented Sep 9, 2016 at 12:51
• quite right. I got a bit carried away. But the argument for the non-existence of the mean is correct.. I will edit my answer. Commented Sep 10, 2016 at 13:01
• Why is it that "there cannot be a mean because there isn't one on the circle"? There's a lot missing in your argument. I'm assuming what you mean by it being the uniform distribution "on the circle" is that $\theta \sim U(-\pi,\pi)$ and $X = \tan(\theta/2)$, but then $\mathbb E[\theta] = 0$ so I don't understand what you're talking about.
Commented Sep 10, 2016 at 13:41
• @jkabrg: I hope the new edits make this more comprehensible Commented Sep 10, 2016 at 14:56

By definition of Lebesgue-Stieltjes integral, the mean exists if:

$$\int \vert x\vert dF(x)<\infty.$$

https://en.wikipedia.org/wiki/Moment_(mathematics)#Significance_of_the_moments

https://en.wikipedia.org/wiki/Lebesgue_integration

It helps to think about such questions from a more abstract point of view.

When we are talking about random variables we implicitly assume the existence of probability space $$(\Omega, \mathcal{F}, \mathbb{P})$$. Then a random variable $$X: \Omega \to \mathbb{R}$$ is simply a measurable function in $$(\mathbb{R}, \mathcal{B})$$. $$X$$ induces a measure $$\mu$$ on the measurable space $$(\mathbb{R}, \mathcal{B})$$ called the pushforward measure defined by $$\mu = \mathbb{P} \circ X^{-1}$$. The pdf of $$X$$ exists if $$\mu$$ is absolutely continuous with respect to the Lebesgue measure on $$(\mathbb{R}, \mathcal{B})$$. The pdf of $$X$$ is a function $$f : \mathbb{R} \to \mathbb{R}$$ and its mean exists if $$\mathbb{E}(X) < \infty$$ where this expectation is the integral of $$X$$ with respect to $$(\Omega, \mathcal{F}, \mathbb{P})$$. Using the law of unconscious statistician we can write $$\mathbb{E}(X) = \int_{\mathbb{R}} x f(x) dx$$ Check out the construction of Lebesgue integral for more details about when $$\mathbb{E}(X) < \infty$$.

• Although this all looks correct, how does it contribute towards the understanding sought in the question?
– whuber
Commented Nov 8, 2020 at 22:06
• My impression is a complete answer to this question should answer two sub-questions: 1. The pdf part of the question - How to think about the pdfs, and 2. How to think about the integral of a pdf and its associated convergence issues. My answer is for the first sub-question. Commented Nov 8, 2020 at 22:57
• The question inquires whether there is some "common trait" to distributions that have undefined means.
– whuber
Commented Nov 9, 2020 at 14:10