4
$\begingroup$

I have come across several usage of the term "heavy tail distributions", but unable to find some good resources in the Internet to answer few of my questions:

  1. Compared to what the tail of the distribution is said to be "heavy"?
  2. Given some data, and their CCDF plots (a log-log plot, say) how do we say its heavy? Simply by curve-fitting with a power-law CCDF?
  3. What is the physical significance (or effect) of the tail being heavy? For example, say the inter-arrival times follow a heavy tail distribution -- what are its implications?
Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
4
$\begingroup$

Wikipedia is often a reasonable start point for basic definitions. In this case there is an entry for heavy-tailed distributions.

The distribution of a random variable $X$ with distribution function $F$ is said to have a heavy right tail if $$\lim_{x\rightarrow\infty} e^{\lambda x}P[X>x]=\infty.$$

for all $\lambda>0$. This can be interpreted as: the tails decay slower than the exponential and this has implications on the existence of moments (see the same wikipedia entry).

Then,

  • (1). To the exponential or compared to an exponential-type behaviour.

  • (2). This can be empirically checked using a normal QQ-plot, for example,

x <- rt(1000,3)

qqnorm(x)

qqline(x,col="red")

As you can see, the lack of linear fit is observed in both tails.

  • (3). An immediate implication of the use of heavy-tailed distribution is that you observe more extreme observations or that the model can capture this sort of behaviour. This is, values far from the shoulders of the distribution. This is reflected in the summary statistics, for example compare the statistics of a normal sample and those of a $t$ sample with $3$ degrees of freedom

    summary(rnorm(1000))

    summary(rt(1000,3))

Improve this answer
$\endgroup$
1
$\begingroup$
  1. What is the physical significance (or effect) of the tail being heavy?

The practical concern is that, when the data come from a heavy-tailed process, there will be occasional extreme data values (or outliers) far from the main body of the data.

  1. Compared to what the tail of the distribution is said to be "heavy"?

The tail of the normal distribution is a good reference point. But see my comments below for caveats.

  1. Given some data, and their CCDF plots (a log-log plot, say) how do we say its heavy? Simply by curve-fitting with a power-law CCDF?

A simple normal quantile-quantile plot will show the extreme values very clearly.

A note on the Wikipedia definitions of heavy tails: They are in some ways useless in that they require unbounded support for a distribution to be heavy-tailed.

However, it is easy (easier, even) to imagine real processes that produce bounded data than it is to imagine real processes that can produce infinitely large (in magnitude) data. And extreme values exist for bounded distributions as well: Take a U(-1,1) distribution, and mix it with a U(-10000, 10000), with mixing probability .0001. The resulting distribution can be called extremely heavy-tailed, capable of producing extreme outliers, but it has finite support.

Thus, alternative definitions of heavy-tailedness are needed for practical purposes. Measures of kurtosis (moment-based and quantile-based) can be useful alternatives.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Your philosophy of "practical purposes" and "real processes" seems like the statistical analog of claiming we should always avoid using Newtonian mechanics because it will fail in the quantum-mechanical and astronomical realms. If we go along with the implicit premise of the question (namely, that we are faced with data that are well-described by a heavy-tailed distribution as defined (say) in Wikipedia), then insisting on "alternative definitions" seems unhelpful. I also don't see the point of using a Normal QQ plot (with its rapidly decreasing tails) to check for exponential tails. $\endgroup$
    – whuber ♦
    Jul 11, 2018 at 22:41
  • $\begingroup$ Seems like unwarranted criticism, whuber. All I asked for was alternative definitions. Do you not agree that my mixture distribution has heavy tails? If yes, then alternative definitions are needed. If no, why not? $\endgroup$
    – BigBendRegion
    Jul 12, 2018 at 15:37
  • $\begingroup$ Yours is the alternative (and idiosyncratic) approach, Peter. So far there is only one actual definition of heavy tails in evidence in this thread--the one on Wikipedia--and I don't see any need to offer another. According to it, your mixture definition does not have heavy tails. My comment was warranted by a concern that neophytes might be confused by reading your post. $\endgroup$
    – whuber ♦
    Jul 12, 2018 at 15:44
  • $\begingroup$ Regarding exponential flavor, my point is again that alternative definitions are needed. Sure, that might be fine for some purposes, but people believe everything they read on Wikipedia, and that is a problem, because the scope of heavy-tailedness defined there is too narrow to define real-world processes that produce extreme values. Maybe another term is needed other than "heavy-tailed"? Such as "distributions capable of producing extreme values"? That is what I am after, but it seems too long. $\endgroup$
    – BigBendRegion
    Jul 12, 2018 at 15:45
  • $\begingroup$ I do not agree that I am idiosyncratic. I think a lot of people are interested in modeling outlier-prone processes. And the definition given in Wikipedia is indeed too narrow for such applications. $\endgroup$
    – BigBendRegion
    Jul 12, 2018 at 15:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.