2
$\begingroup$

I know that one of the standard measures for the "tailedness" of a distribution is kurtosis, i.e. fourth standardized central moment $\frac{\mu_4}{\sigma^4}$. This measure is sort of intuitive to me: as the 4th power "punishes" the outliers much more than 2nd power of the variance does, we would expect a distribution with a heavy tail to have a large kurtosis.

Recently, I thought of using another measure of tailedness--computing the probability content "far away" from the mean, let's call this measure $T$ (for "tail"). Then $$T(X) = \mathbb{P}(\lvert X - \mu\rvert\geq\sigma)$$ where $X$ is a random variable whose tailedness we are measuring, with $\mu = \operatorname{E}X$, and $\sigma^2 = \operatorname{Var}(X)$

We could also perhaps use different multiples of $\sigma$ in the definition (which would, however, also bound the value of $T$ to less than $1$ because of Chebyshev's inequality).

Has this measure been used for judging the tailedness of a distribution? What are the properties causing it not to be in (widespread) use? It seems to me like it would be a good way to capture the "Amazon business model phenomenon," where they can exploit the high overall probability content among niche titles:

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ Because $\Pr(|X-\mu|\gt \sigma)$ is given by the survival function of the standardized variable $|(X-\mu)/\sigma|,$ you can find related definitions of "heavy" or "long" or "fat" tails based on the asymptotic behavior of the survival function. See this search. $\endgroup$ – whuber♦ Sep 29 '18 at 15:48
  • $\begingroup$ You wrote "and $\sigma=\text{Var}(X)$". Did you mean to write either "and $\sigma^{2}=\text{Var}(X)$" or "and $\sigma=\sqrt{\text{Var}(X)}$"? $\endgroup$ – Alexis Oct 2 '18 at 16:57
2
$\begingroup$

It is intuitive as you say. But, I don't think it is more informative than the kurtosis because probabilities of such inequalities don't give much hint about the distribution after the boundaries. To be more specific, let's call $Z=\frac{|X-\mu|}{\sigma}$, and calculate $P(Z\geq k)$ as $0.01$. Density of $Z$ after $k$ could be either focused near $k$, or far away from it. In either case, you'd have the same probability. However, $E[Z^4]$ would immensely differ.

|cite|improve this answer
$\endgroup$
3
$\begingroup$

One attribute of the statistic $P(|X - \mu| \ge \sigma)$ is that it is only giving a single snapshot of "tailedness" that may capture what we want when comparing very specific distributions, but is very misleading for others. As an example, consider a Bernoulli with $p = 0.5$. By this metric, it has a maximally heavy tail, yet most would not consider this a heavy tailed distribution.

As you noted, this statistic can be adjusted by changing $P(|X - \mu| \ge \sigma)$ to $P(|X - \mu| \ge \lambda \sigma)$. Note that if we make $\lambda = 1 + \epsilon$ instead of $\lambda = 1$, our Bernoulli goes from the heaviest possible tails (by this metric) to the lightest possible tails. This suggests we would need to consider this metric over a range of $\lambda$'s to have meaningful insight into our distribution rather than just a particular value of $\lambda$.

With that said, evaluating $P(|X - \mu| \ge \lambda \sigma)$ for the values $\lambda = 1, 2, 3,...$ may help provide some interpretable measures of how the tails of a distribution decay, especially given that most analysts are aware that this metric is approximately 0.32, 0.05 and 0.003 for the normal distribution with $\lambda = 1,2,3$.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ It's unclear what "this statistic" refers to: initially I thought you were talking about kurtosis, but a closer analysis of the rest of this post suggests you are referring the the tail probability in the question. Since a range of $\lambda$ is contemplated in the question, it's unclear what your middle paragraph is addressing. As far as "bit hard to easily comprehend" goes, the resulting function is just the survival function of the size of the standardized variable, a rather familiar object and one routinely used to study distributional tails. $\endgroup$ – whuber♦ Oct 2 '18 at 17:16
  • 1
    $\begingroup$ @whuber: I tried to clarify things a bit. And after a little thought, I do agree that it can provide helpful insight over a range of values. I was thinking of trying to combine those grid of values down to a univariate metric, which would probably loose a lot of interpretability. $\endgroup$ – Cliff AB Oct 2 '18 at 18:31
  • 1
    $\begingroup$ Why not use kurtosis as that univariate metric? Let $\kappa$ denote (non-excess) kurtosis. One connection between kurtosis and Cliff's suggested combination of the grid of values into a univariate metric is $\kappa \ge \sup_\lambda \lambda^4 P(|X-\mu|) > \lambda \sigma).$ Another connection, even more directly related to Cliff's suggestion, is $\sum_{\lambda=1}^\infty [\lambda^4 - (\lambda-1)^4] P(|X-\mu|) \ge \lambda \sigma) \le \kappa \le 1 + \sum_{\lambda=1}^\infty [(\lambda+1)^4 - \lambda^4] P(|X-\mu|) \ge \lambda \sigma).$ $\endgroup$ – Peter Westfall Oct 7 '18 at 17:55
2
$\begingroup$

Heavier-tailed distributions do not necessarily have "more probability in the extreme tails," as is sometimes thought. See here for a counterexample.https://math.stackexchange.com/a/2510884/472987

It is the rare, extreme values that characterize heavy tails. The Bernoulli(p) distribution provides a good example: The smaller the p, the heavier the tail, in the sense that the rare event is more extreme, as measured by number of standard eviations from the mean.

Kurtosis is a good measure of tail heaviness (quantile-based for infinite moments, but for actual data, the ordinary Pearson kurtosis works).

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ The meaning of your first paragraph is unclear. You seem to interpret "heavy tailed" in the sense of large kurtosis rather than in the more established senses described elsewhere here on CV and on Wikipedia, for which your statement would be incorrect. Could you clarify what you mean by "heavy tailed"? $\endgroup$ – whuber♦ Sep 29 '18 at 16:21
  • $\begingroup$ I would say that "heavy tailed" means "capable of producing rare, extreme data values" in the case of a pdf, and "exhibiting rare, extreme data values" in the case of data. "Extreme" can be interpreted as "extreme relative to what a normal distribution produces." In that sense, the Wikipedia definition of heavy-tailedness is unnecessarily restrictive to characterize models for real processes that produce rare, extreme values. Kurtosis types of measures are more general as measures of heavy-tailedness. $\endgroup$ – Peter Westfall Sep 30 '18 at 14:04
  • $\begingroup$ Okay, thank you for the clarification. But I must say it sounds an awful lot like you are defining "heavy tailed" as some version of having "more probability in the extreme tails" (relative to a Normal distribution)! $\endgroup$ – whuber♦ Sep 30 '18 at 15:04
  • $\begingroup$ Yes, definitely, "relative to the extreme tails normal distribution." But that is different from considering a sequence of distirbutions where heaviness of tails increases, and then saying that $P(|X-\mu| > k\sigma)$ must also increase, as the OP seems to suggest. $\endgroup$ – Peter Westfall Sep 30 '18 at 18:23
  • $\begingroup$ I'm unable to read the question as suggesting that, because it's a mathematical impossibility (assuming by "increase" you mean "is an increasing function of $k$") and does not correspond to the graphic in the question. But the reason I was asking is you seem now to have flatly contradicted the opening statement in your post. This renders it less than clear. $\endgroup$ – whuber♦ Sep 30 '18 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.