15
$\begingroup$

From readings about heavy-, and long-tailed distributions, I understood that all long-tailed distributions are heavy-tailed, but not all heavy-tailed distributions are long-tailed.

Could somebody please provide an example of:

  • a continuous, symmetric, zero-mean density function that is long-tailed
  • a continuous, symmetric, zero-mean density function that is heavy-tailed but not long-tailed

so I can better understand the meaning of their definitions?

It would be even better if both could have a unit variance.

$\endgroup$
4

1 Answer 1

19
$\begingroup$

The two definitions are close, but not exactly the same. One difference lies in the need for the survival ratio to have a limit.

For most of this answer I will ignore the criteria for the distribution to be continuous, symmetric, and of finite variance, because these are easy to accomplish once we have found any finite-variance heavy-tailed distribution that is not long-tailed.


A distribution $F$ is heavy-tailed when for any $t\gt 0$,

$$\int_\mathbb{R} e^{t x} dF(x) = \infty.\tag{1}$$

A distribution with survival function $G_F = 1-F$ is long-tailed when

$$\lim_{x\to \infty} \frac{G_F(x+1)}{G_F(x)} = 1.\tag{2}$$

Long-tailed distributions are heavy. Furthermore, because $G$ is nonincreasing, the limit of the ratio $(2)$ cannot exceed $1$. If it exists and is less than $1$, then $G$ is decreasing exponentially--and that will allow the integral $(1)$ to converge.

The only way to exhibit a heavy-tailed distribution that is not long-tailed, then, is to modify a long-tailed distribution so that $(1)$ continues to hold while $(2)$ is violated. It's easy to screw up a limit: change it in infinitely many places that diverge to infinity. That will take some doing with $F$, though, which must remain increasing and cadlag. One way is to introduce some upward jumps in $F$, which will make $G$ jump downwards, lowering the ratio $G_F(x+1)/G_F(x)$. To this end, let's define a transformation $T_u$ that turns $F$ into another valid distribution function while creating a sudden jump at the value $u$, say a jump halfway from $F(u)$ to $1$:

$$T_u[F](x) = \begin{cases} F(x) & u<x \\ \frac{1}{2} (1-F(x))+F(x) & u\geq x \end{cases}$$

This alters no basic property of $F$: $T_u[F]$ is still a distribution function.

The effect on $G_F$ is to make it drop by a factor of $1/2$ at $u$. Therefore, since $G$ is non-decreasing, then whenever $u-1 \le x \lt u$,

$$\frac{G_{T_u[F]}(x + 1)}{G_{T_u[F]}(x )} \le \frac{1}{2}.$$

If we pick an increasing and diverging sequence of $u_i$, $i=1, 2, \ldots$, and apply each $T_{u_i}$ in succession, it determines a sequence of distributions $F_i$ with $F_0=F$ and

$$F_{i+1} = T_{u_i}[F_i]$$

for $i \ge 1$. After the $i^\text{th}$ step, $F_i(x), F_{i+1}(x), \ldots$ all remain the same for $x\lt u_i$. Consequently the sequence of $F_i(x)$ is a nondecreasing, bounded, pointwise sequence of distribution functions, implying its limit

$$F_\infty = \lim_{i\to\infty} F_i$$

is a distribution function. By construction, it is not long-tailed because there are infinitely many points at which its survival ratio $G_{F_\infty}(x+1)/G_{F_\infty}(x))$ drops to $1/2$ or below, showing it cannot have $1$ as a limit.

Figure 1: An altered survival function

This plot shows a survival function $G(x) = x^{-1/5}$ that has been cut down in this manner at points $u_1 \approx 12.9, u_2 \approx 40.5, u_3 \approx 101.6, \ldots.$ Note the logarithmic vertical axis.

The hope is to be able to choose $(u_i)$ so that $F_\infty$ remains heavy-tailed. We know, because $F$ is heavy-tailed, that there are numbers $0 = u_0 \lt u_1 \lt u_2 \lt \cdots \lt u_n \cdots$ for which

$$\int_{u_{i-1}}^{u_i} e^{x/i} dF(x) \ge 2^{i-1}$$

for every $i \ge 1$. The reason for the $2^{i-1}$ on the right is that the probabilities assigned by $F$ to values up to $u_i$ have been successively cut in half $i-1$ times. That procedure, when $dF(x)$ is replaced by $dF_{j}(x)$ for any $j\ge i$, will reduce $2^{i-1}$ to $1$, but no lower.

Figure 2: A cut-down density function

This is a plot of $x f(x)$ for densities $f$ corresponding to the previous survival function and its "cut down" version. The areas under this curve contribute to the expectation. The area from $1$ to $u_1$ is $1$; the area from $u_1$ to $u_2$ is $2$, which when cut down (to the lower blue portion) becomes an area of $1$; the area from $u_2$ to $u_3$ is $4$, which when cut down becomes an area of $1$, and so on. Thus, the area under each successive "stair step" to the right is $1$.

Let us pick such a sequence $(u_i)$ to define $F_\infty$. We can check that it remains heavy-tailed by picking $t=1/n$ for some whole number $n$ and applying the construction:

$$\eqalign{ \int_\mathbb{R} e^{t x} dF_\infty(x) &=\int_\mathbb{R} e^{x/n} dF_\infty(x) \\ &= \sum_{i=1}^\infty \int_{u_{i-1}}^{u_i} e^{x/n} dF_\infty(x) \\ &\ge \sum_{i=n+1}^\infty \int_{u_{i-1}}^{u_i} e^{x/n} dF_\infty(x) \\ &\ge \sum_{i=n+1}^\infty \int_{u_{i-1}}^{u_i} e^{x/i} dF_\infty(x) \\ &= \sum_{i=n+1}^\infty \int_{u_{i-1}}^{u_i} e^{x/i} dF_i(x) \\ &\ge \sum_{i=n+1}^\infty 1, }$$

which still diverges. Since $t$ is arbitrarily small, this demonstrates that $F_\infty$ remains heavy-tailed, even though its long-tailed property has been destroyed.

Figure 3: plot of G(1+x)/G(x)

This is a plot of the survival ratio $G(x+1)/G(x)$ for the cut down distribution. Like the ratio of the original $G$, it tends toward an upper accumulation value of $1$--but for unit-width intervals terminating at the $u_i$, the ratio suddenly drops to only half of what it originally was. These drops, although becoming less and less frequent as $x$ increases, occur infinitely often and therefore prevent the ratio from approaching $1$ in the limit.


If you would like a continuous, symmetric, zero-mean, unit-variance example, begin with a finite-variance long-tailed distribution. $F(x) = 1 - x^{-p}$ (for $x \gt 0$) will do, provided $p \gt 1$; so would a Student t distribution for any degrees of freedom exceeding $2$. The moments of $F_\infty$ cannot exceed those of $F$, whence it too has finite variance. "Mollify" it via convolution with a nice smooth distribution, such as a Gaussian: this will make it continuous but will not destroy its heavy tail (obviously) nor the absence of a long tail (not quite as obvious, but it becomes obvious if you change the Gaussian to, say, a Beta distribution whose support is compact).

Symmetrize the result--which I will still call $F_\infty$--by defining

$$F_s(x) = \frac{1}{2}\left(1 + \text{sgn}(x) F_\infty(|x|)\right)$$

for all $x\in\mathbb{R}$. Its variance will remain finite, so it can be standardized to the desired distribution.

$\endgroup$
3
  • 2
    $\begingroup$ Brilliantly explained. You offered not just an example but also the justification for it. The clarity of the explanation allowed me to understand (almost) the whole of it. I will practice it in some numerical examples. $\endgroup$
    – toliveira
    Sep 1, 2015 at 16:56
  • $\begingroup$ Nice notation that I can use to ask you re (1), can we say because of (1) that exponentiation distribution is the reference point for heavy tail distributions? $\endgroup$ Feb 25, 2021 at 19:29
  • 1
    $\begingroup$ @Easy That's not a bad intuition: the probability has to decrease faster than exponentially in order for that integral (1) to be bounded. It is then natural to wonder "what's so special about exponential decrease?" An answer perhaps is found in contemplating the usefulness of the moment generating function (mgf): (1) is equivalent to stating that the mgf does not exist for positive values of $t,$ which implies the moments of $F$ cannot be obtained as the Taylor coefficients of the mgf at $t=0.$ $\endgroup$
    – whuber
    Feb 25, 2021 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.