13
$\begingroup$

Under a standard gaussian distribution (mean 0 and variance 1), the kurtosis is $3$. Compared to a heavy tail distribution, is the kurtosis normally larger or smaller?

$\endgroup$
  • 1
    $\begingroup$ does en.wikipedia.org/wiki/Kurtosis#Excess_kurtosis answer your question? $\endgroup$ – Ben Bolker Jul 29 at 2:18
  • $\begingroup$ Partially, I am wondering if there exists an intuitive way to think about it $\endgroup$ – user321627 Jul 29 at 2:21
  • 7
    $\begingroup$ Please explain what you mean by a "heavy tail distribution." Some people have different interpretations of this term. Depending on the interpretation, the answer is either yes or no, so this matters. $\endgroup$ – whuber Jul 29 at 15:18
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/10726/… $\endgroup$ – kjetil b halvorsen Jul 29 at 17:54
  • 2
    $\begingroup$ short answer is "typically, but not always" $\endgroup$ – Glen_b Aug 3 at 2:36
15
+500
$\begingroup$

I. A direct answer to the OP

Answer: It depends on what you mean by “heavy tails.” By some definitions of “heavy tails,” the answer is “no,” as pointed out here and elsewhere.

Why do we care about heavy tails? Because we care about outliers (substitute the phrase “rare, extreme observation” if you have a problem with the word “outlier.” However, I will use the term “outlier” throughout for brevity.) Outliers are interesting from several points of view: In finance, outlier returns cause much more money to change hands than typical returns (see Taleb‘s discussion of black swans). In hydrology, the outlier flood will cause enormous damage and needs to be planned for. In statistical process control, outliers indicate “out of control” conditions that warrant immediate investigation and rectification. In regression analysis, outliers have enormous effects on the least squares fit. In statistical inference, the degree to which distributions produce outliers has an enormous effect on standard t tests for mean values. Similarly, the degree to which a distribution produces outliers has an enormous effect on the accuracy of the usual estimate of the variance of that distribution.

So for various reasons, there is a great interest in outliers in data, and in the degree to which a distribution produces outliers. Notions of heavy-tailedness were therefore developed to characterize outlier-prone processes and data.

Unfortunately, the commonly-used definition of “heavy tails” involving exponential bounds and asymptotes is too limited in its characterization of outliers and outlier-prone data generating processes: It requires tails extending to infinity, so it rules out bounded distributions that produce outliers. Further, the standard definition does not even apply to a data set, since all empirical distributions are necessarily bounded.

Here is an alternative class of definitions of ”heavy-tailedness,” which I will call “tail-leverage($m$)” to avoid confusion with existing definitions of heavy-tailedness, that addresses this concern.

Definition: Assume absolute moments up to order $m>2$ exist for random variables $X$ and $Y$. Let $U = |(X - \mu_X)/\sigma_X|^m$ and let $V =|(Y - \mu_Y)/\sigma_Y|^m$. If $E(V) > E(U)$, then $Y$ is said to have greater tail-leverage($m$) than $X$.

The mathematical rationale for the definition is as follows: Suppose $E(V) > E(U)$, and let $\mu_U = E(U)$. Draw the pdf (or pmf, in the discrete case, or in the case of an actual data set) of $V$, which is $p_V(v)$. Place a fulcrum at $\mu_U$ on the horizontal axis. Because of the well-known fact that the distribution balances at its mean, the distribution $p_V(v)$ “falls to the right” of the fulcrum at $\mu_U$. Now, what causes it to “fall to the right”? Is it the concentration of mass less than 1, corresponding to the observations of $Y$ that are within a standard deviation of the mean? Is it the shape of the distribution of $Y$ corresponding to observations that are within a standard deviation of the mean? No, these aspects are to the left of the fulcrum, not to the right. It is the extremes of the distribution (or data) of $Y$, in one or both tails, that produce high positive values of $V$, which cause the “falling to the right.”

BTW, the term “leverage” should now be clear, given the physical representation involving the fulcrum. But it is worth noting that, in the characterization of the distribution “falling to the right,” that the “tail leverage” measures can legitimately be called measures of “tail weight.” I chose not to do that because the "leverage" term is more precise.

Much has been made of the fact that kurtosis does not correspond directly to the standard definition of “heavy tails.” Of course it doesn’t. Neither does it correspond to any but one of the infinitely many definitions of “tail leverage” I just gave. If you restrict your attention to the case where $m=4$, then an answer to the OP’s question is as follows:

Greater tail leverage (using $m=4$ in the definition) does indeed imply greater kurtosis (and conversely). They are identical.

Incidentally, the “leverage” definition applies equally to data as it does to distributions: When you apply the kurtosis formula to the empirical distribution, it gives you the estimate of kurtosis without all the so-called “bias corrections.” (This estimate has been compared to others and is reasonable, often better in terms of accuracy; see "Comparing Measures of Sample Skewness and Kurtosis," D. N. Joanes and C. A. Gill, Journal of the Royal Statistical Society. Series D (The Statistician) Vol. 47, No. 1 (1998), pp. 183-189.)

My stated leverage definition also resolves many of the various comments and answers given in response to the OP: Some beta distributions can be more greatly tail-leveraged (even if “thin-tailed” by other measures) than the normal distribution. This implies a greater outlier potential of such distributions than the normal, as described above regarding leverage and the fulcrum, despite the normal distribution having infinite tails and the beta being bounded. Further, uniforms mixed with classical “heavy-tailed” distributions are still "heavy-tailed," but can have less tail leverage that the normal distribution, provided the mixing probability on the “heavy tailed” distribution is sufficiently low so that the extremes are very uncommon, and assuming finite moments.

Tail leverage is simply a measure of the extremes (or outliers). It differs from the classic definition of heavy-tailedness, even though it is arguably a viable competitor. It is not perfect; a notable flaw is that it requires finite moments, so quantile-based versions would be useful as well. Such alternative definitions are needed because the classic definition of “heavy tails” is far too limited to characterize the universe of outlier-prone data-generating processes and their resulting data.

II. My paper in The American Statistician

My purpose in writing the paper “Kurtosis as Peakedness, 1905-2014: R.I.P.” was to help people answer the question, “What does higher (or lower) kurtosis tell me about my distribution (or data)?” I suspected the common interpretations (still seen, by the way), “higher kurtosis implies more peaked, lower kurtosis implies more flat” were wrong, but could not quite put my finger on the reason. And, I even wondered that maybe they had an element of truth, given that Pearson said it, and even more compelling, that R.A. Fisher repeated it in all revisions of his famous book. However, I was not able to connect any math to the statement that higher (lower) kurtosis implied greater peakedness (flatness). All the inequalities went in the wrong direction.

Then I hit on the main theorem of my paper. Contrary to what has been stated or implied here and elsewhere, my article was not an “opinion” piece; rather, it was a discussion of three mathematical theorems. Yes, The American Statistician (TAS) does often require mathematical proofs. I would not have been able to publish the paper without them. The following three theorems were proven in my paper, although only the second was listed formally as a “Theorem.”

Main Theorem: Let $Z_X = (X - \mu_X)/\sigma_X$ and let $\kappa(X) = E(Z_X^4)$ denote the kurtosis of $X$. Then for any distribution (discrete, continuous or mixed, which includes actual data via their discrete empirical distribution), $E\{Z_X^4 I(|Z_X| > 1)\}\le\kappa(X)\le E\{Z_X^4 I(|Z_X| > 1)\} +1$.

This is a rather trivial theorem to prove but has major consequences: It states that the shape of the distribution within a standard deviation of the mean (which ordinarily would be where the “peak” is thought to be located) contributes very little to the kurtosis. Instead, the theorem implies that for all data and distributions, kurtosis must lie within $\pm 0.5$ of $E\{Z_X^4 I(|Z_X| > 1)\} + 0.5$.

A very nice visual image of this theorem by user "kjetil b Halvorsen" is given at https://stats.stackexchange.com/a/362745/102879; see my comment that follows as well.

Figure

The bound is sharpened in the Appendix of my TAS paper:

Refined Theorem: Assume $X$ is continuous and that the density of $Z_X^2$ is decreasing on [0,1]. Then the “+1” of the main theorem can be sharpened to “+0.5”.

This simply amplifies the point of the main theorem that kurtosis is mostly determined by the tails.

A third theorem proven in my TAS paper states that large kurtosis is mostly determined by (potential) data that are $b$ standard deviations away from the mean, for arbitrary $b$.

Theorem 3: Consider a sequence of random variables $X_i$,$ i = 1,2,\dots$, for which $\kappa(X_i) \rightarrow \infty$. Then $E\{Z_i^4I(|Z_i| > b)\}/ \kappa(X_i) \rightarrow 1$, for each $b>0$.

The third theorem states that high kurtosis is mostly determined by the most extreme outliers; i.e., those observations that are $b$ or more standard deviations from the mean.

These are mathematical theorems, so there can be no argument with them. Supposed “counterexamples” given in this thread and in other online sources are not counterexamples; after all, a theorem is a theorem, not an opinion.

So what of one suggested “counterexample” appearing in this thread, where spiking the data with many values at the mean (which thereby increases “peakedness”) causes greater kurtosis? Actually, that example just makes the point of my theorems: When spiking the data in this way, the variance is reduced, thus the observations in the tails are more extreme, in terms of number of standard deviations from the mean. And it is observations with large standard deviation from the mean, according to the theorems in my TAS paper, that cause high kurtosis. It’s not the peakedness. Or to put it another way, the reason that the spike increases kurtosis is not because of the spike itself, it is because the spike causes a reduction in the standard deviation, which makes the tails more standard deviations from the mean (i.e., more extreme), which in turn increases the kurtosis.

It simply cannot be stated that higher kurtosis implies greater peakedness, because you can have a distribution that is perfectly flat over an arbitrarily high percentage of the data (pick 99.99% for concreteness) with infinite kurtosis. (Just mix a uniform with a Cauchy suitably; there are some minor but trivial and unimportant technical details regarding how to make the peak absolutely flat.) By the same construction, high kurtosis can be associated with any shape whatsoever for 99.99% of the central distribution - U-shaped, flat, triangular, multi-modal, etc.

There is also a suggestion in this thread that the center of the distribution is important, because throwing out the central data of the Cauchy example in my TAS paper makes the data have low kurtosis. But this is also due to outliers and extremes: In throwing out the central portion, one increases the variance so that the extremes are no longer extreme (in terms of $Z$ values), hence the kurtosis is low.

All supposed "counterexamples" given in this thread and in other online sources actually obey my theorems. Theorems have no counterexamples; otherwise, they would not be theorems.

A more interesting exercise than “spiking” or “deleting the middle” is this: Take the distribution of a random variable $X$ (discrete or continuous, so it includes the case of actual data), and replace the mass/density within one standard deviation of the mean arbitrarily, but keep the mean and standard deviation of the resulting distribution the same as that of $X$.

Q: How much change can you make to the kurtosis statistic over all such possible replacements?

A: The difference between the maximum and minimum kurtosis values over all such replacements is $\le 0.25. $

The above question and its answer comprise yet another theorem. Anyone want to publish it? I have its proof written down (it’s quite elegant, as well as constructive, identifying the max and min distributions explicitly), but I lack the incentive to submit it as I am now retired. I have also calculated the actual max differences for various distributions of $X$; for example, if $X$ is normal, then the difference between the largest and smallest kurtosis is over all replacements of the central portion is 0.141. Hardly a large effect of the center on the kurtosis statistic!

On the other hand, if you keep the center fixed, but replace the tails, keeping the mean and standard deviation constant, you can make the kurtosis infinitely large. Thus, the effect on kurtosis of manipulating the center while keeping the tails constant, is $\le 0.25$. On the other hand, the effect on kurtosis of manipulating the tails, while keeping the center constant, is infinite.

So, while yes, I agree that spiking a distribution at the mean does increase the kurtosis, I do not find this helpful to answer the question, “What does higher kurtosis tell me about my distribution?” There is a difference between “A implies B” and “B implies A.” Just because all bears are mammals does not imply that all mammals are bears. Just because spiking distribution increases kurtosis does not imply that increasing kurtosis implies a spike; see the uniform/Cauchy example alluded to above in my answer.

It is precisely this faulty logic that caused Pearson to make the peakedness/flatness interpretations in the first place. He saw a family of distributions for which the peakedness/flatness interpretations held, and wrongly generalized. In other words, he observed that a bear is a mammal, and then wrongly inferred that a mammal is a bear. Fisher followed suit forever, and here we are.

A case in point: People see this picture of "standard symmetric PDFs" (on Wikipedia at https://en.wikipedia.org/wiki/File:Standard_symmetric_pdfs.svg) and think it generalizes to the “flatness/peakedness” conclusions.

Wikipedia image

Yes, in that family of distributions, the flat distribution has the lower kurtosis and the peaked one has the higher kurtosis. But it is an error to conclude from that picture that high kurtosis implies peaked and low kurtosis implies flat. There are other examples of low kurtosis (less than the normal distribution) distributions that are infinitely peaked, and there are examples of infinite kurtosis distributions that are perfectly flat over an arbitrarily large proportion of the observable data.

The bear/mammal conundrum also arises in the Finucan conditions, which state (oversimplified) that if tail probability and peak probability increase (losing some mass in between to maintain the standard deviation), then kurtosis increases. This is all fine and good, but you cannot turn the logic around and say that increasing kurtosis implies increasing tail and peak mass (and reducing what is in between). That is precisely the fatal flaw with the sometimes-given interpretation that kurtosis measures the “movement of mass simultaneously to the tails and peak but away from the shoulders." Again, all mammals are not bears. A good counterexample to that interpretation is given here https://math.stackexchange.com/a/2523606/472987 in “counterexample #1, which shows a family of distributions in which the kurtosis increases to infinity, while the mass inside the center stays constant. (There is also a counterexample #2 that has the mass in the center increasing to 1.0 yet the kurtosis decreases to its minimum, so the often-made assertion that kurtosis measures “concentration of mass in the center” is wrong as well.) Many people think that higher kurtosis implies “more probability in the tails.” This is not true; counterexample #1 shows that you can have higher kurtosis with less tail probability when the tails extend.

So what does kurtosis measure? It precisely measures tail leverage (which can be called tail weight as well) as amplified through fourth powers, as I stated above with my definition of tail-leverage($m$).

I would just like to reiterate that my TAS article was not an opinion piece. It was instead a discussion of mathematical theorems and their consequences. There is much additional supportive material in the current post that has come to my attention since writing the TAS article, and I hope readers find it to be helpful for understanding kurtosis.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – whuber Aug 5 at 19:15
  • $\begingroup$ @PeterWestfall , I believe that we can improve your refined theorem. The “+1” of the main theorem can be sharpened to “+$1/3$”. I have added an answer related to that here math.stackexchange.com/a/3781761 $\endgroup$ – Sextus Empiricus Aug 6 at 9:45
1
$\begingroup$

Heavy Tails or "Peakedness"?

Kurtosis is usually thought of as denoting heavy tails; however, many decades ago, statistics students were taught that higher kurtosis implied more "peakedness" versus the normal distribution.

The Wikipedia page (suggested in a comment) does note this in saying that higher kurtosis usually comes from (a) more data close to the mean with rare values very far from the mean, or (b) heavy tails in the distribution.

A Thin-Tailed High-Kurtosis Example

Usually, these two situations occur at the same time. However, a simple example shows a light-tailed distribution with high kurtosis.

The beta distribution has very light tails: the tails are literally bounded in that they cannot extend past 0 or 1. However, the following $R$ code generates a beta distribution with high kurtosis:

n.rv <- 10000  
rv <- rbeta(n.rv, 1, 0.1)  
z <- (rv - mean(rv))/sd(rv)  # standardized rv for kurtosis calculation
kurt <- sum(z^4)/(n.rv-2)    # plenty of debate on the right df; not crucial here

Running this simulation gives a kurtosis of 9 to 10. (The exact value would be 9.566, to three decimal places.)

But What About a Heavy-Tailed Distribution?

You asked, however, about heavy-tailed distributions -- and for some intuition.

In general, heavier-tailed distributions will have higher kurtoses.

The Intuition

To intuitively see this, consider two symmetric pdfs $f_X,f_Y$ that are standardized: $E(X)=E(Y)=0$ and ${\rm var}(X)={\rm var}(Y)=1$. Let's also say these densities have support on the whole real line, so $f_X,f_Y>0$ everywhere.

Let's assume the contributions toward kurtosis from the centers of the densities are similar: $E(X^4|-k\leq X\leq k)\approx E(Y^4|-k\leq Y\leq k)$ for some finite $k$. Since these distributions both have probability density > 0 in their tails (getting out toward $\pm\infty$), we can see that their kurtoses ($E(X^4),E(Y^4)$) will likely be dominated by the contribution from $X,Y$ approaching $\pm\infty$.

This would not be true would be if the tails decayed very quickly: quicker than exponentially and quicker than even $e^{-x^2}$. However, you said this is in comparison to a Gaussian pdf, so we know the Gaussian tails die off like $f_X\propto e^{-x^2}$. Since the heavier-tailed distribution has tails that are thicker (ie do not die off as quickly), we know those tails will contribute more to $E(Y^4)$

Issues

As you can tell (if you read the comments), there are plenty of counterexamples to the general guidelines you are trying to get. Kurtosis is far less well understood than, say, variance. In fact, it is not even clear what it the best estimator for kurtosis.

What is the Correct Estimator?

For small samples, Cramér (1957) suggested replacing $\frac{1}{n-2}$ with $\frac{n^2-2n+3}{(n-1)(n-2)(n-3)}$ and subtracting $\frac{3(n-1)(2n-3)}{n(n-2)(m-3)}\hat\sigma^4$ and Fisher (1973) suggested replacing $\frac{1}{n-2}$ with $\frac{n(n+1)}{(n-1)(n-2)(n-3)}$. (Fisher's justification of unbiasedness under normality, however, is odd for a centered moment which is of most interest for non-normal distributions.)

Contributions from the Center of the Distribution

The center of the distribution can also have a large effect on the kurtosis. For example, consider a power-law variable: a variable having a density with tails decaying on the order of $|x|^{-p}$. ($p>5$ so that the kurtosis is finite.) These are clearly "fat-tailed" since the tails decay slower than $e^{-x^2}$ (and even $e^{-x}$). Despite that, mixtures of uniform and power-law random variables can have kurtoses less than 3 (i.e. negative excess kurtoses).

Variance of Variance?

More recently, I have heard people talk about kurtosis as the "variance of variance" (or "vol of vol" in mathematical finance). That idea makes more sense since many types of data exhibit heteroskedasticity or different regimes with different variances. For a great example, just look at a historical plot of US unemployment: the numbers reported remained within a relatively tight range until they exploded due to a pandemic and stay-at-home orders.

Are the very high unemployment observations something we would typically expect? Or, are they due to a change in the regime of the macroeconomy? Either way, the resulting series has very high kurtosis and the answer for why may affect what we consider to be reasonable modeling assumptions in the future.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ "Kurtosis tells you virtually nothing about the shape of the peak - its only unambiguous interpretation is in terms of tail extremity, that is, either existing outliers (for the sample kurtosis) or propensity to produce outliers (for the kurtosis of a probability distribution)." Direct quite from the much-recommended article by Westfall (2014, "Kurtosis as Peakedness, 1905–2014. R.I.P.", The American Statistician). $\endgroup$ – Stephan Kolassa Jul 29 at 5:30
  • 2
    $\begingroup$ No problem. Have you read the article? No, TAS is not JASA, but they do have standards, and Westfall gives cogent (and, to me, convincing) arguments. My (and, I suppose, Westfall's) beef is that the association of kurtosis with peakedness continues to be propagated, although it has almost no foundation. I do not believe that mentioning kurtosis and peakedness together is of any use, but it runs the danger of reinforcing this conventional non-wisdom. So I will continue to interject the Westfall article when I see the two terms together, in the hope of eventually breaking this association. $\endgroup$ – Stephan Kolassa Jul 29 at 7:07
  • $\begingroup$ I can't determine what this post is saying because we still lack any definition of "heavy" tails: what definition are you assuming? $\endgroup$ – whuber Jul 29 at 15:20
  • 5
    $\begingroup$ Irving Kaplansky (better known for later work in algebra) showed back in 1945 in JASA that higher and lower kurtosis can arise from any combination of lower/higher peak and differing tail weight. Nevertheless I think Peter Westfall's emphasis on kurtosis as measuring tail weight is much more helpful than talking about peakedness (although I doubt he would like that summary). It's vacuous but crucial that kurtosis is whatever kurtosis measures is the only completely safe summary. $\endgroup$ – Nick Cox Jul 29 at 16:50
  • 1
    $\begingroup$ Unfortunately, your statement following "The only way this would not be true..." is incorrect. The tails only need to decay faster than $|x|^{-5}.$ A simple class of counterexamples is obtained through mixtures of uniform and power-law variables: by putting enough weight on the uniform component you can make the excess kurtosis negative, even though the tails are heavy (and fat and long) in the usual senses of the literature on extreme events. $\endgroup$ – whuber Jul 29 at 19:41
1
$\begingroup$

The question being asked is "In comparison with a standard gaussian random variable, does a distribution with heavy tails have higher kurtosis?"

The simulations below show what happens when you make the tails heavier, using different definitions of tails. (Define the "tails section" of a normal distribution, and then add values to that section, so you have a scenario EXACTLY matching the question being asked.)

First I add a few values 3 standard deviations and beyond to a random normal distribution with 100 initial values. (R code is shown; mean = 0, SD = 1.)

> AA <- rnorm(100,0,1)
> kurtosis(AA)
[1] 2.899004
> AA1 <- c(AA,3,3.5,4,5,6,-3,-3.5,-4,-5,-6)
> kurtosis(AA1)
[1] 6.868121

The kurtosis is signficantly higher, and the distribution is notably "pointier."

enter image description here

Next, I took a random normal distribution with 1,000 items, sorted it, and then doubled the items in the top 1/9th and bottom 1/9th (their occurrence, not their values). This resulted in the kurtosis dropping from 3.15 to 2.55, and a notably "flatter" distribution.

> library(moments)
> library(groupdata2)
> V1 <- rnorm(1000,0,1)
> kurtosis(V1)
[1] 3.14978
> W1 <- sort(V1)
> X1 <- group(W1, 9)
> X1a <- subset(X1, X1$.groups == 1)
> X1b <- subset(X1, X1$.groups == 9)
> Z1 <- c(V1, X1a$data, X1b$data)
> kurtosis(Z1)
[1] 2.545745

enter image description here

Using 15 groups instead, and doubling the 1st and 15th reduces the kurtosis to 2.73, and also makes the distribution flatter.

Next, I used Groups = 40 to only increase the tails at 2 standard deviations and beyond. I used a random normal distribution with 10 million values. The initial kurtosis was 2.9981. Repeatedly adding the values in groups 1 and 40 back into the the distribution results in an initial slight increase, and then drops in kurtosis: 3.0256, 2.914, 2.7758, 2.6429; which leads to a trimodal distribution. (If you smooth this out, you get a result like Groups=9 above; lower kurtosis.)

Next, I used Groups = 667 to only increase the tails at 3 standard deviations and beyond. (Same 10 million distribution.) As you add the values of groups 1 and 667 back in, the kurtosis increases, and then starts falling at the 19th iteration: 3.15 3.29 3.40 ... 3.95093 3.95153 3.95015. The histogram shows the distribution at the 19th step.

enter image description here

The results will depend on your distribution, how you are making the tails heavier, and whether you are changing something in addition to the tails.


Kurtosis can be calculated as the average of Z^4 for all data points, where Z is the common z-score: z=(x−x̅)/σ. "The average value of z is always zero, but the average value of z^4 is always ≥ 1, and is larger when you have a few big deviations on either side of the mean than when you have a lot of small ones." https://brownmath.com/stat/shape.htm#Kurtosis

This means that kurtosis is simply the relationship between the number of values close to the mean compared to the number of values in the outlying tail.

This is demonstrated with the following R code.

> a1 <- c(-.0001, -.001,-.1,-.1,0,0,0,0,.1,.1,.001, .0001)
> kurtosis(a1)
[1] 2.999697

Now, add four 0's to the middle, making it "pointier."

> a1 <- c(-.0001, -.001,-.1,-.1,0,0,0,0,0,0,0,0,.1,.1,.001, .0001)
> kurtosis(a1)
[1] 3.999596

Now, remove the four most outlying values from the first set, making it "flatter."

> a1 <- c(-.1,-.1,0,0,0,0,.1,.1)
> kurtosis(a1)
[1] 2

NOTE: The R function I'm using uses 3 to indicate normal kurtosis.


The following statements use a kurtosis function that used 0 as normal kurtosis.

The following are typically found:

  • "Positive values of kurtosis indicate that a distribution is peaked and possess thick tails. Leptokurtic distributions have positive kurtosis values."
  • "Negative values of kurtosis indicate that a distribution is flat and has thin tails. Platykurtic distributions have negative kurtosis values."
  • https://www.simplypsychology.org/kurtosis.html

You have to be careful** about interpreting a kurtosis value you read:

  • There are different ways to calculate kurtosis.
  • Some kurtosis functions return 0 for a normal distribution, so 3 is very high. (The value is "standardized" to zero.)
  • Some kurtosis functions return 3 for a normal distribution, so 3 is normal.
  • At least one kurtosis function returns sqrt(2/pi) or .7979 for a normal distribution.
  • You have to read the documentation for the function you are using.

https://www.itl.nist.gov/div898/handbook/eda/section3/eda35b.htm
https://cran.r-project.org/web/packages/moments/moments.pdf

Whatever is used as normal with a given function, "positive" (leptokurtic) is above that, and "negative" (platykurtic) is below that.



Regarding "tails only" argument being discussed here

Personally, I find it argumentative to talk about obscure, statistically useless distributions such as 0,0,0,0,0,0,0,250 as an example of why kurtosis only talks about the tails. That's what at least one site making this argument does. The fact is that with the vast majority of distributions kurtosis will be used on, you have some midpoint that is higher than the tails... And thus the shape of high point and the tails are bound.

I reviewed the paper by Westfall - "Kurtosis as Peakedness, 1905 – 2014. R.I.P." The arguments are severely misrepresenting the picture. Look at the main point, the Cauchy distribution in Figure 1. Westfall goes on to argue that the substantial peak plays an insignificant role in the kurtosis number, using some forumulas.

What a magnificent example of getting numbers to say what you want.

The peak is ENTIRELY RESPONSIBLE for the kurtosis of 437. You remove the values from that radical peak, so the top sits just above it's neighbors (in normal fashion), and you get a relatively normal kurtosis. (!!!) You didn't touch the tails.

Westfall declares "peadkedness" dead. Really? Whoever said that kurtosis "defines" the shape of the peak? I guess I missed that. And whoever said you can't get different shapes with the same kurtosis? I guess I missed that, too. The fact is that the peak and tail extremes are bound, and common observations of themes follow. THAT is what I've heard. Also, I have already seen the counter-examples. They've been around forever. -- People are going to generalize about the peaks, because generalizing has its valid place.

Also, the simulations I show at the beginning of this post show that the
kurtosis doesn't predict the shape, thickness, or width of the tails - insofar as generalizations go. Isn't that Westfall's complaint about the peaks? Exactly. In reality, generalizations can be made about both, and is seems the generalizations about peakness are actually EASIER.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – whuber Aug 5 at 19:15
0
$\begingroup$

In comparison with a standard gaussian random variable, does a distribution with heavy tails have higher kurtosis?

A short and simple answer:

It is not necessary for a distribution with heavy tails to have a higher kurtosis than a standard gaussian random variable. (one exception is when you define heavy tails as the distribution being leptokurtic)

Heavy tails defined in terms of the rate of decrease to infinity

Many definitions for heavy tails have some definition that relate to the rate at which the tails of a distribution (with infinite support) fall of to zero. For instance wikipedia "heavy-tailed distributions are probability distributions whose tails are not exponentially bounded". For these type of definitions it is the case that: if you scale the weight of the tails, (e.g. by mixing with another distribution with less dominant tails), then the tails will still have the same rate and limiting behavior.

If a distribution has finite kurtosis, then it can be any value independent from the type of tails (any value above 1, which is the limit for all distributions). Heavy or not, the type of tail does not dictate some minimum kurtosis (except when it is infinite or undefined).

Say, if some heavy tail distribution has kurtosis x>3, then you can 'decrease it' by mixing it with a non-heavy tail distribution that has kurtosis<3 (but the tails still remain heavy, they are only scaled with a factor). Only when you have infinite kurtosis, these tails matter (ie. you can not remove the infinity by diluting the heavy tail distribution by mixing with another distribution).

Heavy tails defined in terms of kurtosis or other moments

Several other answers have mentioned a definition of tails in terms of moments. In that case the above reasoning does not apply.

Some of those answers define a heavy tail in terms of 'kurtosis > 3' in which case the question becomes a tautology (as whuber noted in the comments).

However, the question still remains whether a distribution with a heavy tail (when it is defined for another higher order moment instead of the kurtosis) must have a higher kurtosis as well.

In this q&a here it is shown that we do not need to have the situation that a higher/lower kurtosis, must also mean that the other moments are equally higher/lower.

Some similar distribution as in that answer with approximately $2.4<a<2.5$ will have higher 6th standardized moment, but lower kurtosis, in comparison to the normal distribution.

$$f(x,a) = \begin{cases} 0.0005 & \text{if} & x = -a \\ 0.2495 & \text{if} & x = -1 \\ 0.5000 & \text{if} & x = 0 \\ 0.2495 & \text{if} & x = 1 \\ 0.0005 & \text{if} & x = a \\ 0 & \text{otherwise} \end{cases}$$

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Odd answer. By mixing, you obtain a different distribution, with different tail properties. $\endgroup$ – Peter Westfall Aug 1 at 10:47
  • $\begingroup$ No, not me. And you are presuming a particular definition of "tail properties." According to the "tail leverage" measures defined in my post, you certainly do get different tail properties. $\endgroup$ – Peter Westfall Aug 1 at 15:42
  • $\begingroup$ No, but the common definition is too limited as I note. It does not even apply to actual data. That is why alternative notions of tailedness are needed. $\endgroup$ – Peter Westfall Aug 1 at 17:56
0
$\begingroup$

If you go with a formal definition, such as one in Wikipedia, then the tails must be heavier than exponential distribution. Exponential distribution's excess kurtosis is 6. Student t distribution's excess kurtosis goes from infinite to zero as the degrees of freedom go from 4 to infinity, and Student t converges to normal. Also, some people, myself included, use a much simpler definition: positive excess kurtosis. So, the answer is yes, excess kurtosis will be positive for heavy tailed distributions.

I can't say whether it is possible to construct a distribution that would satisfy formal requirements of heavy tailed distribution and has negative excess kurtosis. If it is possible, I bet it would be a purely theoretical construct that nobody uses to model heavy tails anyway.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ This strikes me as a circular argument: what is your definition of "heavier" and how is that related to kurtosis? $\endgroup$ – whuber Jul 29 at 15:17
  • $\begingroup$ positive excess kurtosis is heavy tails. Gaussian has zero excess kurtosis. there are more nuanced definitions about heavy and fat tails around exponential tails etc. i prefer a simple notion of excess kurtosis $\endgroup$ – Aksakal Jul 29 at 16:01
  • 2
    $\begingroup$ If that's the case, then the question is tautological and the answer is trivial. A more common (and far more interesting) concept of heavy tail is the one at Wikipedia. $\endgroup$ – whuber Jul 29 at 16:05
  • $\begingroup$ I agree that the question is tautological. The Wiki article is good, but from my vantage point they make things more complicated than in practice with distinctions between fat and heavy tailed for instance, which isn't even a convention (which they dont fail to mention) $\endgroup$ – Aksakal Jul 29 at 16:09
  • 2
    $\begingroup$ I am trying to suggest that when a question appears trivial or tautological, there's probably an alternative interpretation that is the intended one. In such cases we should attempt explicitly to apply the non-trivial interpretation in our answers or, barring that, to post comments asking for clarification. $\endgroup$ – whuber Jul 29 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.