Skip to main content
Cross Validated

Return to Answer

Fixed a couple typos, noted the mixture distributions which yield kurtoses < 3
Source Link
kurtosis
kurtosis
  • 1.6k
  • 5
  • 16

What is the Correct Estimator?

For small samples, Cramér (1957) suggested replacing $\frac{1}{n-2}$ with $\frac{n^2-2n+3}{(n-1)(n-2)(n-3)}$ and subtracting $\frac{3(n-1)(2n-3)}{n(n-2)(m-3)}\hat\sigma^4$ and Fisher (1973) suggested replacing $\frac{1}{n-2}$ with $\frac{n(n+1)}{(n-1)(n-2)(n-3)$$\frac{n(n+1)}{(n-1)(n-2)(n-3)}$. (Fisher's justification of unbiasedness under normality, however, is odd for a centered moment which is of most interest for non-normal distributions.)

Contributions from the Center of the Distribution

The center of the distribution can also have a large effect on the kurtosis. For example, consider a power-law variable: a variable having a density with tails decaying on the order of $|x|^{-p}$. ($p>5$ so that the kurtosis is finite.) These are clearly "fat-tailed" since the tails decay slower than $e^{-x^2}$ (and even $e^{-x}$). Despite that, mixtures of uniform and power-law random variables can have kurtoses less than 3 (i.e. negative excess kurtoses).

Variance of Variance?

More recently, I have heard people talk about kurtosis as the "variance of variance" which(or "vol of vol" in mathematical finance). That idea makes more sense since many types of data exhibit heteroskedasticity or different regimes with different variances. For a great example, just look at a historical plot of US unemployment: the numbers reported remained within a relatively tight range until they exploded due to a pandemic and stay-at-home orders.

Are the very high unemployment observations something we would typically expect? Or, are thethey due to a change in the regime of the macroeconomy? Either way, the resulting series has very high kurtosis and the answer for why may affect what we consider to be reasonable modeling assumptions in the future.

For small samples, Cramér (1957) suggested replacing $\frac{1}{n-2}$ with $\frac{n^2-2n+3}{(n-1)(n-2)(n-3)}$ and subtracting $\frac{3(n-1)(2n-3)}{n(n-2)(m-3)}\hat\sigma^4$ and Fisher (1973) suggested replacing $\frac{1}{n-2}$ with $\frac{n(n+1)}{(n-1)(n-2)(n-3)$. (Fisher's justification of unbiasedness under normality, however, is odd for a centered moment which is of most interest for non-normal distributions.

More recently, I have heard people talk about kurtosis as the "variance of variance" which makes more sense since many types of data exhibit heteroskedasticity or different regimes with different variances. For a great example, just look at a historical plot of US unemployment: the numbers reported remained within a relatively tight range until they exploded due to a pandemic and stay-at-home orders.

Are the very high unemployment observations something we would typically expect? Or, are the due to a change in the regime of the macroeconomy? Either way, the resulting series has very high kurtosis and the answer for why may affect what we consider to be reasonable modeling assumptions in the future.

What is the Correct Estimator?

For small samples, Cramér (1957) suggested replacing $\frac{1}{n-2}$ with $\frac{n^2-2n+3}{(n-1)(n-2)(n-3)}$ and subtracting $\frac{3(n-1)(2n-3)}{n(n-2)(m-3)}\hat\sigma^4$ and Fisher (1973) suggested replacing $\frac{1}{n-2}$ with $\frac{n(n+1)}{(n-1)(n-2)(n-3)}$. (Fisher's justification of unbiasedness under normality, however, is odd for a centered moment which is of most interest for non-normal distributions.)

Contributions from the Center of the Distribution

The center of the distribution can also have a large effect on the kurtosis. For example, consider a power-law variable: a variable having a density with tails decaying on the order of $|x|^{-p}$. ($p>5$ so that the kurtosis is finite.) These are clearly "fat-tailed" since the tails decay slower than $e^{-x^2}$ (and even $e^{-x}$). Despite that, mixtures of uniform and power-law random variables can have kurtoses less than 3 (i.e. negative excess kurtoses).

Variance of Variance?

More recently, I have heard people talk about kurtosis as the "variance of variance" (or "vol of vol" in mathematical finance). That idea makes more sense since many types of data exhibit heteroskedasticity or different regimes with different variances. For a great example, just look at a historical plot of US unemployment: the numbers reported remained within a relatively tight range until they exploded due to a pandemic and stay-at-home orders.

Are the very high unemployment observations something we would typically expect? Or, are they due to a change in the regime of the macroeconomy? Either way, the resulting series has very high kurtosis and the answer for why may affect what we consider to be reasonable modeling assumptions in the future.

Added an Issues section; added more regularity conditions for tail decay to dominate
Source Link
kurtosis
kurtosis
  • 1.6k
  • 5
  • 16

Let's assume the contributions toward kurtosis from the centers of the densities are similar: $E(X^4|-k\leq X\leq k)\approx E(Y^4|-k\leq Y\leq k)$ for some finite $k$. Since these distributions both have probability density > 0 in their tails (getting out toward $\pm\infty$), we can see that their kurtoses ($E(X^4),E(Y^4)$) will likely be dominated by the contribution from $X,Y$ approaching $\pm\infty$.

The only way thisThis would not be true would be if the tails decayed very quickly: quicker than exponentially and quicker than even $e^{-x^2}$. However, you said this is in comparison to a Gaussian pdf, so we know the Gaussian tails die off like $f_X\propto e^{-x^2}$. Since the heavier-tailed distribution has tails that are thicker (ie do not die off as quickly), we know those tails will contribute more to $E(Y^4)$

Issues

As you can tell (if you read the comments), there are plenty of counterexamples to the general guidelines you are trying to get. Kurtosis is far less well understood than, say, variance. In fact, it is not even clear what it the best estimator for kurtosis.

For small samples, Cramér (1957) suggested replacing $\frac{1}{n-2}$ with $\frac{n^2-2n+3}{(n-1)(n-2)(n-3)}$ and subtracting $\frac{3(n-1)(2n-3)}{n(n-2)(m-3)}\hat\sigma^4$ and Fisher (1973) suggested replacing $\frac{1}{n-2}$ with $\frac{n(n+1)}{(n-1)(n-2)(n-3)$. (Fisher's justification of unbiasedness under normality, however, is odd for a centered moment which is of most interest for non-normal distributions.

More recently, I have heard people talk about kurtosis as the "variance of variance" which makes more sense since many types of data exhibit heteroskedasticity or different regimes with different variances. For a great example, just look at a historical plot of US unemployment: the numbers reported remained within a relatively tight range until they exploded due to a pandemic and stay-at-home orders.

Are the very high unemployment observations something we would typically expect? Or, are the due to a change in the regime of the macroeconomy? Either way, the resulting series has very high kurtosis and the answer for why may affect what we consider to be reasonable modeling assumptions in the future.

Since these distributions both have probability density > 0 in their tails (getting out toward $\pm\infty$), we can see that their kurtoses ($E(X^4),E(Y^4)$) will likely be dominated by the contribution from $X,Y$ approaching $\pm\infty$.

The only way this would not be true would be if the tails decayed very quickly: quicker than exponentially and quicker than even $e^{-x^2}$. However, you said this is in comparison to a Gaussian pdf, so we know the Gaussian tails die off like $f_X\propto e^{-x^2}$. Since the heavier-tailed distribution has tails that are thicker (ie do not die off as quickly), we know those tails will contribute more to $E(Y^4)$

Let's assume the contributions toward kurtosis from the centers of the densities are similar: $E(X^4|-k\leq X\leq k)\approx E(Y^4|-k\leq Y\leq k)$ for some finite $k$. Since these distributions both have probability density > 0 in their tails (getting out toward $\pm\infty$), we can see that their kurtoses ($E(X^4),E(Y^4)$) will likely be dominated by the contribution from $X,Y$ approaching $\pm\infty$.

This would not be true would be if the tails decayed very quickly: quicker than exponentially and quicker than even $e^{-x^2}$. However, you said this is in comparison to a Gaussian pdf, so we know the Gaussian tails die off like $f_X\propto e^{-x^2}$. Since the heavier-tailed distribution has tails that are thicker (ie do not die off as quickly), we know those tails will contribute more to $E(Y^4)$

Issues

As you can tell (if you read the comments), there are plenty of counterexamples to the general guidelines you are trying to get. Kurtosis is far less well understood than, say, variance. In fact, it is not even clear what it the best estimator for kurtosis.

For small samples, Cramér (1957) suggested replacing $\frac{1}{n-2}$ with $\frac{n^2-2n+3}{(n-1)(n-2)(n-3)}$ and subtracting $\frac{3(n-1)(2n-3)}{n(n-2)(m-3)}\hat\sigma^4$ and Fisher (1973) suggested replacing $\frac{1}{n-2}$ with $\frac{n(n+1)}{(n-1)(n-2)(n-3)$. (Fisher's justification of unbiasedness under normality, however, is odd for a centered moment which is of most interest for non-normal distributions.

More recently, I have heard people talk about kurtosis as the "variance of variance" which makes more sense since many types of data exhibit heteroskedasticity or different regimes with different variances. For a great example, just look at a historical plot of US unemployment: the numbers reported remained within a relatively tight range until they exploded due to a pandemic and stay-at-home orders.

Are the very high unemployment observations something we would typically expect? Or, are the due to a change in the regime of the macroeconomy? Either way, the resulting series has very high kurtosis and the answer for why may affect what we consider to be reasonable modeling assumptions in the future.

Source Link
kurtosis
kurtosis
  • 1.6k
  • 5
  • 16

Heavy Tails or "Peakedness"?

Kurtosis is usually thought of as denoting heavy tails; however, many decades ago, statistics students were taught that higher kurtosis implied more "peakedness" versus the normal distribution.

The Wikipedia page (suggested in a comment) does note this in saying that higher kurtosis usually comes from (a) more data close to the mean with rare values very far from the mean, or (b) heavy tails in the distribution.

A Thin-Tailed High-Kurtosis Example

Usually, these two situations occur at the same time. However, a simple example shows a light-tailed distribution with high kurtosis.

The beta distribution has very light tails: the tails are literally bounded in that they cannot extend past 0 or 1. However, the following $R$ code generates a beta distribution with high kurtosis:

n.rv <- 10000  
rv <- rbeta(n.rv, 1, 0.1)  
z <- (rv - mean(rv))/sd(rv)  # standardized rv for kurtosis calculation
kurt <- sum(z^4)/(n.rv-2)    # plenty of debate on the right df; not crucial here

Running this simulation gives a kurtosis of 9 to 10. (The exact value would be 9.566, to three decimal places.)

But What About a Heavy-Tailed Distribution?

You asked, however, about heavy-tailed distributions -- and for some intuition.

In general, heavier-tailed distributions will have higher kurtoses.

The Intuition

To intuitively see this, consider two symmetric pdfs $f_X,f_Y$ that are standardized: $E(X)=E(Y)=0$ and ${\rm var}(X)={\rm var}(Y)=1$. Let's also say these densities have support on the whole real line, so $f_X,f_Y>0$ everywhere.

Since these distributions both have probability density > 0 in their tails (getting out toward $\pm\infty$), we can see that their kurtoses ($E(X^4),E(Y^4)$) will likely be dominated by the contribution from $X,Y$ approaching $\pm\infty$.

The only way this would not be true would be if the tails decayed very quickly: quicker than exponentially and quicker than even $e^{-x^2}$. However, you said this is in comparison to a Gaussian pdf, so we know the Gaussian tails die off like $f_X\propto e^{-x^2}$. Since the heavier-tailed distribution has tails that are thicker (ie do not die off as quickly), we know those tails will contribute more to $E(Y^4)$