Westfall says, "the proportion of the kurtosis that is determined by the central $\mu\pm\sigma$ range is usually quite small" but is the reverse true?

Question

In his article that debunks the notion of kurtosis as measuring distribution peakedness, Peter Westfall writes,

[T]he proportion of the kurtosis that is determined by the central $\mu\pm\sigma$ range is usually quite small.

I read this to mean that we learn little about kurtosis by knowing how much of the density is within a standard deviation of the mean.

Is the reverse true? Knowing the kurtosis, can we say anything (beyond the Chebyshev inequality) about how much density is contained within one standard deviation of the mean?

(I think I am happy to work with empirical distributions, in order to avoid issues with undefined means and infinite variance.)

Reference

Westfall, Peter H. "Kurtosis as peakedness, 1905–2014. RIP." The American Statistician 68.3 (2014): 191-195.

Answer 1

Answer edited 9/15/2021:

In his answer to the OP, @whuber claims as follows:

For a distribution with kurtosis $\kappa$, the total density within one SD of the mean lies between $1−1/\kappa$ and $1$, where $\kappa$ is the (non-excess) kurtosis of the distribution.

THIS CLAIM IS FALSE.

The following example shows clearly that @whuber's result is false.

Consider my "Counterexample #1" from here, with $\theta = .001.$ In that counterexample, the kurtosis is $25.5,$ the range $1-1/\kappa$ to $1.0$ is from $0.96$ to $1.0,$ yet the probability within a standard deviation of the mean is $0.5$. These statements are verified by the R code:

th = .001
Z = c(-sqrt(.155/th +1.44), -1.2, -.5, +.5, +1.2, +sqrt(.155/th +1.44))
p = c(th/2, (.5-th)/2, .25, .25, (.5-th)/2, th/2)

sum(p)       # The probabilities sum to one so it is a valid pmf
sum(Z*p)     # The mean is zero
sum(Z^2*p)   # The variance is one

plot(Z, p, type="h", lwd = 4, cex.lab=1.5, cex.axis=1.5, 
        ylab="Probability")
abline(v=c(-1,1), lty=2, lwd=2)  # Shows values within +- 1 sd

k = sum(Z^4*p)
k       # Kurtosis is 25.5

range = c(1 - 1/k,1)
range     # (.96, 1.0) is the range suggested by @whuber's false theorem 
          # about probability within a sd of mu

sum(p[abs(Z)<1])  # 0.5 is the actual probability within +- 1sd

Here is a graph of the counterexample distribution. The dashed vertical lines mark the $\mu \pm \sigma$ limits, within which it is clearly visible that there is only $0.50$ probability.

You can also illustrate the counterexample using a reproducible data set and summary statistics. The following R code generates $1000000$ samples from the counterexample distribution, a large enough sample size so that the "bias corrections" are negligible. The estimated kurtosis is $26.02$, the range $(1 - 1/26.02, 1)$, within which the central probability is supposed to lie, is $(.96,1)$, yet the estimated central probability is $0.4999$.

set.seed(12345)
N = 1000000
Data = sample(Z, N, p, replace = TRUE)
xbar = mean(Data)
s = sd(Data)

library(moments)
ku = kurtosis(Data)
ku
c(1-1/ku, 1)  # @whuber's false claim of central probability range

sum( Data >= xbar -s & Data <= xbar +s )/N  # Actual central probability

It is amusing to see just how spectacularly @whuber's result does fail. In my counterexample #1 family of distributions, the kurtosis can tend to infinity, implying, according to @whuber's "result," that the central probability approaches $1.0$. But instead, the central probability stays constant at $0.5$!

One does not need to construct fancy counterexamples to illustrate such spectacular failure of @whuber's claim. Consider the common $T_\nu$ distribution, the Student T distribution with degrees of freedom parameter $\nu$. For $\nu > 4$, its mean is zero, its variance is $\sigma^2 = \nu/(\nu -2)$, and its (non-excess) kurtosis is $\kappa = 6/(\nu-4) +3$. In the range $4 < \nu \le 5$, the kurtosis ranges from $9$ to $\infty$, while the probability within $\pm \sigma$ can be calculated numerically, in R notation, as

pt(sigma, nu) - pt(-sigma,nu)

The following R code and resulting graph shows the range claimed by @whuber (dashed black lines), along with the actual central probability (solid red line).

nu = seq(4.0001, 4.9999, .0001)
sigma = sqrt(nu/(nu-2))
kurt = 6/(nu-4) + 3
Cent.Prob = pt(sigma, nu) - pt(-sigma, nu)

Upper.Bound = rep(1, length(nu))
Lower.Bound = 1 - 1/kurt
plot(nu, Cent.Prob, ylim = c(.6,1), type="l", col="red", 
   ylab="Central Probability", xlab = "degrees of freedom")
points(nu, Upper.Bound, type="l", lty=2)
points(nu, Lower.Bound, type="l", lty=2)

Again, there is a spectacular failure of @whuber's claim, in that the claim implies the central probability must be essentially $1.0$ (for $\nu \approx 4$), when in fact it is far less (around $0.77$).

Thus, @whuber's claim is false: The central probability need not lie in @whuber's stated range. In fact, as my Counterexample #1 shows, the central probability need not increase at all with larger kurtosis.

Here are two results that shed additional light on the relation of kurtosis to the center.

Theorem 1. Consider a random variable $X$ (includes data via the empirical distribution) that has, wlog, mean = 0, variance = 1, and finite fourth moment. Now, create a new random variable $X'$ by replacing the mass/density of $p_X$ within $0 \pm 1$ arbitrarily, but maintaining $E(X')=0$ and $Var(X')=1.$ Then the difference between the maximum and minimum kurtosis statistics over all such replacements is less than 0.25.

Theorem 2. Consider a random variable $X$ as in Theorem 1. Now, create a new random variable $X'$ by replacing the mass/density of $p_X$ outside of $0 \pm 1$ arbitrarily, but maintaining $E(X')=0$ and $Var(X')=1$ in such replacements. Then the difference between the maximum and minimum kurtosis statistics over all such replacements is unbounded (i.e., infinite).

Thus, the effect of moving mass near the center has at most a very small effect on kurtosis, while the effect of moving mass in the tails has an infinite effect.

While one is trying to prove a theorem that proves that the center somehow is related to kurtosis, it is very helpful to know in advance what counterexamples may exist to such a theorem.

Good counterexamples are given here.

"Counterexample #1" shows a family of distributions in which the kurtosis increases to infinity, while the mass inside $\mu \pm \sigma$ stays a constant 0.5.

"Counterexample #2" shows a family of distributions where the mass within $\mu \pm \sigma$ increases to 1.0, yet the kurtosis decreases to its minimum.

So the often-made assertion that kurtosis measures “concentration of mass in the center” is obviously wrong.

Many people think that higher kurtosis implies “more probability in the tails.” This is not true either: Counterexample #1 shows that you can have higher kurtosis with less tail probability when the tails extend.

Instead, kurtosis precisely measures tail leverage. See

How the kurtosis value can determine the unhealthy event

and

In comparison with a standard gaussian random variable, does a distribution with heavy tails have higher kurtosis? .

Answer 2

For a distribution with kurtosis $\kappa,$ the total density within one SD of the mean lies between $1-1/\kappa$ and $1.$

The intuition behind this is twofold: (1) kurtosis is most heavily influenced by extreme values and (2) it is more influenced by such extremes than the standard deviation. The question concerns what can be said about the proportion of "non-extreme" values based on the value $\kappa$ of the kurtosis alone, where "non-extreme" is taken to be within one standard deviation of the mean. The statement above shows that the kurtosis does determine some non-trivial limits on that proportion. It is the tightest such result that can be expressed in general: in the demonstrations below I offer two families of distributions (the simplest possible: each is supported on just three values) showing how the bounds can be approached arbitrarily closely (or even attained) for any kurtosis.

---

Let's begin with some simplifications. Because kurtosis is a standardized central moment, we may (with no loss of generality) restrict our study to zero-mean, unit-variance ("standardized") distributions. In these cases the kurtosis equals the (raw) fourth moment and the probability of a non-extreme value is the probability of the interval $[-1,1].$

The power-mean inequality asserts the fourth root of $\kappa$ is no less than the standard deviation. Thus, $\kappa \ge 1.$

Maximizing the chance of non-extreme values, given $\kappa$

Consider a standardized "trinomial" distribution supported on the values $-1, b,$ and $a$ with probabilities $p-r,$ $1-p,$ and $r,$ respectively where $|b|\le 1$ and $a \gt 1.$ Because probabilities are non-negative, $0 \le r \le p \le 1.$ The figure displays three examples with various values of the kurtosis.

Our conditions on the moments yield three equations:

$$\left\{\begin{aligned} 0 &= (p-r)(-1) + (1-p)(b) + r(a) &= r-p + (1-p)b + ra &\quad \text{Mean is }0 \\ 1 &= (p-r)(-1)^2 + (1-p)(b)^2 + r(a)^2 &= p-r + (1-p)b^2 + ra^2 &\quad \text{Variance is } 1 \\ \kappa &= (p-r)(-1)^4 + (1-p)(b)^4 + r(a)^4 &= p-r + (1-p)b^4 + ra^4 &\quad \text{Kurtosis.} \end{aligned}\right.$$

Because the chance of a non-extreme value is $(p-r) + (1-p) = 1-r,$ we seek to minimize $r$ given $\kappa,$ subject to these constraints.

No matter what $\kappa$ might be, the supremum of these chances is $1,$ because the infimum of $r$ is zero.

To show this, we need to demonstrate that for sufficiently and arbitrarily small values of $r,$ there are values of $p,$ $b,$ and $a$ satisfying the three constraints. Because $r$ can be taken to be arbitrarily small--it is, in effect, an infinitesimal--let's solve these equation to first order in $r.$ That's straightforward to do, giving

$$a \cong \left(\frac{\kappa - 1}{r}\right)^{1/4},\quad b \cong 1 - \left(\frac{a^2-1}{2}\right)r,\quad p = \frac{r(1+a) + b}{b+1}.$$

The value of $a$ grows large as $r$ diminishes, but in such a way that $ra$ and $ra^2$ continue to grow small with $r.$ When $r$ is sufficiently small, then, $0\lt r\lt p\lt 1$ and (clearly) $b \lt 1$ and $a\gt 1,$ assuring all constraints are satisfied. This completes the demonstration that $r$ can be made arbitrarily small for any $\kappa \gt 1.$

The figure displays numerical solutions for a range of $\kappa.$

Minimizing the chance of non-extreme values, given $\kappa$

This example is inspired by the proof of Chebyshev's Inequality. Recall the proof proceeds by replacing $|x|^s$ by $a^4$ times the indicator of the function $|x| \ge a$ for some constant $a,$ whence

$$\mu_s = E[|X|^s]\ \ge\ E[a^s \mathcal{I}(|X|\ge a)] = a^s \Pr(|X|\ge a).$$

This is Chebyshev's Inequality.

We may limit the search for a minimum to symmetric distributions. This is because we may take any standardized distribution with distribution function $F$ and symmetrize it to the distribution function $x\to (F(x) + 1 - F(-x))/2$ without changing any of the constraints or the objective function.

Suppose, then, $F$ is any discrete symmetric standardized distribution with kurtosis $\kappa \gt 1.$ If the event $(-\sqrt{\kappa},\sqrt{\kappa})$ has any positive probability let it do so at the values $\pm a.$ Change $F$ by zeroing the probability at $\pm a$ and compensate by increasing the probabilities at $\pm\sqrt{\kappa}$ by $a^2/(2\kappa)$ and increasing the probability at $0$ by $1 - a^2/\kappa.$ A quick calculation establishes this preserves the variance, kurtosis, and symmetry of $F,$ because the change in any moment of order $s$ is

$$\delta \mu_s = 2p\left[(a^2/\kappa)(\sqrt{\kappa}^s) - a^s\right] = 2pa^2\left[\kappa^{s/2 - 1} - a^{s-2}\right]$$

and this equals zero when $a^{s-2} = \kappa^{s/2-1},$ which always includes the solutions $s=2$ and $s=4.$

Consequently, after applying this operation to every one of the (at most countable) values with positive probability in the interval $(-\sqrt{\kappa}, \sqrt{\kappa}),$ we may assume all the probability within that interval (equal to $1-r,$ say) is concentrated at $0.$ Chebyshev's inequality (applied to the $s=4^\text{th}$ moment) tells us this probability must be at least $1-1/(\kappa^{1/4})^4 = 1-1/\kappa.$ Consequently $1-r \ge 1 - 1/\kappa.$ This minimum is attained (as seen in the proof of Chebyshev's inequality) by the trinomial distribution assigning probabilities $1/(2\kappa)$ to the values $\pm \sqrt{\kappa}$ and all the remaining probability to $0.$ Here are plots of three such distributions.

Finally, the discrete distributions are dense within the space of all distributions (this says nothing other than the graph of any CDF may be arbitrarily well approximated by at most a countable set of points strategically positioned along it). Because all the functionals involved (moments and probabilities) are continuous properties of the distribution, these results hold for all distributions.