Thin tails and the Generalized Pareto

Rewriting my question:

On this Mathworks page:

https://www.mathworks.com/help/stats/generalized-pareto-distribution.html

it is said (as many textbooks say) that

"Distributions whose tails decrease exponentially, such as the normal, lead to a generalized Pareto shape parameter of zero."

I don't understand this statement. The tail of an exponential will decrease as exp(-x) while the tail of a normal will decrease as exp(-x^2).

Can someone explain this?

• I don't see how the conclusion "cannot approximate" follows from "support ... is infinite." For instance, the original formulation of the famous Central Limit Theorem approximates the standard Normal distribution with a standardized Binomial distribution, whose support not only is bounded but also is finite (that is, discrete). Perhaps your meaning of "approximate" depends on the sense in which you want to use one distribution to approximate another: could you explain?
– whuber
Commented Sep 5, 2019 at 20:27
• Okay, let me try to answer. I'm interested in approximating the thin tail of a distribution that extends out to infinity. It appears to me that the GP can't do that for me, since the shape parameter for a thin-tailed distribution doesn't give a reduced form that actually extends to infinity. This seems very different from your point that focusses on the central core of the distribution. Commented Sep 5, 2019 at 20:32
• The CLT does not focus on any part of the distribution: it's a statement about convergence of the entire distribution. (See stats.stackexchange.com/questions/3734 for instance.) The issue comes down to how you intend to measure the discrepancy between a distribution and its approximator: could you disclose that to us? In case you haven't thought about the question in this way, see stats.stackexchange.com/questions/86429 or stats.stackexchange.com/questions/168851 for some ideas and examples of how people compare distributional tails.
– whuber
Commented Sep 5, 2019 at 20:50
• That is my interpretation, but I need confidence, so a source that explains this issue (in the context of the generalized Pareto) would be welcome. I can't find one that I can understand. Commented Sep 6, 2019 at 16:00
• Something else I've learned, which takes me to the crux of my confusion (modeling thin-tailed distributions), apparently the GP distribution is not a complete approximation of all tails, but, instead, only a description of a "wide class" of distributions. This is probably obvious to experts in extreme-value theory, but I can tell you many scientists don't appreciate this limitation. In fact, many scientists think you can blindly use the GP to model pretty much all extreme tails. I'm about done, here. Thanks to WHuber. Commented Sep 6, 2019 at 20:11

At https://stats.stackexchange.com/a/86503/919 I have argued that to study the right tail of a distribution $$F,$$ you should analyze its survival function $$1-F$$ for large values of the argument. Because such questions often compare tail behavior to some form of exponential, it is convenient to look at the logarithm of the survival function.

Background and definitions

The Generalized Pareto Distribution is a family of distributions. Their shapes depend on one real parameter $$\xi.$$ The other two parameters allow you to select any unit of measurement you like by choosing its origin $$\mu$$ and scale $$\sigma,$$ and therefore do not affect asymptotic tail behavior. Consequently we may set $$\mu=0$$ and $$\sigma=1$$ without losing any generality.

For $$\xi\ne 0,$$ the log survival function is

$$\log(1 - F_\xi(z)) = -\frac{\log(1 + \xi z)}{\xi}\tag{1}$$

on the domain $$[0, \infty)$$ for positive $$\xi$$ and the domain $$[0, -1/\xi)$$ for negative $$\xi.$$

The case of negative $$\xi$$ is relatively uninteresting because there is no probability of values exceeding $$-1/\xi:$$ the right tail ultimately is zero. Let's focus henceforth on non-negative $$\xi.$$

The "edge case" is the limit as $$\xi\to 0.$$ L'Hopital's Rule enables us to find this limit as

$$\lim_{\xi\to 0} \log\left(1-F_\xi(z)\right) = - \lim_{\xi\to 0} \frac{\frac{d}{d\xi}(\log(1+\xi z))}{\frac{d}{d\xi}\xi} = -\lim_{\xi\to 0} \frac{z}{1+\xi z} = -z.$$

Thus, it makes sense to include the distribution with log survival function $$-z$$ in this family. That distribution is the Exponential distribution.

Analysis

Now we fix $$\xi\ge 0$$ and study the distribution for large $$z.$$ In this case $$(1)$$ is asymptotically a power tail because the Taylor expansion of the logarithm around $$1$$ shows

\eqalign{-\frac{1}{\xi}\log(1+\xi z) &= -\frac{1}{\xi}\left(\log(\xi) + \log(z) + \log\left(1 + \frac{1}{\xi z}\right)\right) \\ &= -\frac{1}{\xi} \log(z) + C_\xi - \frac{1}{\xi^2}z^{-1} + O(z^{-2}) }

where $$C_\xi = -(\log(\xi))/\xi$$ is just a normalizing constant. All the terms eventually become arbitrarily small compared to the first, which is a constant times $$\log(z):$$ when exponentiated, this shows the survival function asymptotically is $$z$$ to the $$-1/\xi$$ power, as claimed.

Consequently, the members of the Generalized Pareto Distribution have three kinds of right tail behavior:

1. The survival function eventually is zero for negative $$\xi.$$

2. The survival function is asymptotic to the $$-1/\xi$$ power for positive $$\xi.$$

3. The survival function is exponential in the sense that it is exactly proportional to $$e^{-z}$$ when $$\xi=0.$$

This does not cover the gamut of possible asymptotic tail behaviors of arbitrary distributions. For instance, the log survival function of any Normal distribution is asymptotic to $$-z^2,$$ which decreases faster than any power or the exponential. Thus, the Generalized Pareto Distribution cannot model just any distribution: it can only model certain distributions with "long" or "heavy" tails. See Example of heavy-tailed distribution that is not long-tailed for more on these concepts.

By definition, a heavy-tailed distribution $$F$$ is one with a tail that decays slower than any exponential in the sense that for all positive constants $$t,$$

$$\int_{\mathbb R} e^{tz} \mathrm{d}F(z) = \infty.$$

When $$\xi=0$$ and $$0\lt t \lt 1,$$

$$\int_{\mathbb R} e^{tz} \mathrm{d}F_\xi(z) =\int_0^\infty e^{tz} e^{-z} \mathrm{d}z = \frac{1}{1-t}\ne\infty$$

shows that

The Exponential distribution ($$\xi=0$$) is not heavy-tailed. However, all Generalized Pareto Distributions with $$\xi\gt 0$$ are heavy-tailed.

In this sense the case $$\xi=0$$ is a boundary between the heavy power tailed Pareto distributions and their limiting non-heavy Exponential distribution. This is the basis for the intuition that this family, for $$\xi\ge 0,$$ is a good one for modeling just the heavy-tailed distributions.

Appendix: Technicalities

If these claims about the tails aren't clear, the following is a rigorous demonstration. Note that for any positive numbers $$x,$$ $$\xi,$$ and $$t$$ for which $$x \gt 2(1+t)/(\xi t^2),$$ the number $$u = \xi t x - 2 = 2/t$$ is positive. Bounding the exponential $$e^u$$ below by the first three terms of its Taylor series $$1+u+u^2/2!+\cdots$$ yields

\eqalign{ \exp(\xi t x-2) &\gt 1 + \xi t x - 2 + \frac{(\xi t x - 2)^2}{2!} \\ &= 1 + \xi x(-t + \xi t^2 x/2) \\ &\gt 1 + \xi x(-t + (1+t)) \\ &= 1 + \xi x. }

Upon taking logs and dividing by $$\xi$$ we obtain

$$t x \gt \frac{2}{\xi} + \frac{1}{\xi} \log(1 + \xi x).$$

For notational convenience write $$a = \max(0, 2(1-t)/(\xi t^2)).$$ Integration by parts, simple bounding arguments, and judicious introduction of the preceding inequality show

\eqalign{ \int_{\mathbb R} e^{tz} \mathrm{d}F_\xi(z) &\ge \int_0^\infty e^{tz} \mathrm{d}F_\xi(z) \\ &= \int_\infty^0 e^{tz}\mathrm{d}(1-F_\xi(z)) \\ &= \lim_{z\to\infty}\left[(e^{tz} (1-F_\xi(z)))\left|_z^0\right. + t\int_0^z e^{tx}(1-F_\xi(x))\mathrm{d}x\right] \\ &\ge t \lim_{z\to\infty}\int_0^z e^{tx}(1-F_\xi(x))\mathrm{d}x \\ &= t \lim_{z\to\infty}\int_0^z \exp\left(tx -\frac{1}{\xi}\log(1+\xi x)\right)\mathrm{d}x \\ &\ge t \lim_{z\to\infty}\int_a^z \exp\left(tx -\frac{1}{\xi}\log(1+\xi x)\right)\mathrm{d}x \\ &\gt t \lim_{z\to\infty}\int_a^z \exp\left(\frac{2}{\xi}\right)\mathrm{d}x \\ &= t \exp\left(\frac{2}{\xi}\right) \lim_{z\to\infty} \int_a^z \mathrm{d}x, }

which diverges, QED.

• Very good. I just added a comment above, with my own recognition of the important point you make: "This does not cover the gamut of possible asymptotic tail behaviors of arbitrary distributions." I would be very good if introductory books on extreme-value theory would emphasize this point. Commented Sep 6, 2019 at 20:14

Maybe the ambiguity is between the tail and the domain of attraction as related to the Fisher-Tippett Gnedenko theorem. The theorem defines three Domains of Attraction (DA): Fréchet, Gumbel and Weibull, and moreover within each of the two DA Weibull and Fréchet, the tails can be ordered according to a tail index $$\xi$$ which corresponds to the shape parameter of a Generalized Extreme Value (GEV) distribution or that of the Generalized Pareto (GP) distribution. While the domain attraction of a distribution is determined by the tail, the converse is not true: a DA contains distributions with different "tail thicknesses" in a reasonable acceptance of this expression.

We can say that two continuous distributions with survival functions $$S_X(x)$$ and $$S_Y(x)$$ are tail-equivalent if the two distributions share the same upper end-point $$\omega$$ (possibly $$\infty$$) and if moreover $$S_X(x) \underset{x \to \omega}{\sim} a \,S_{Y}(x)$$ for some finite positive number $$a$$. If a distribution is tail-equivalent to the GEV or GP distribution with shape $$\xi$$ , then it belongs to the domain of attraction of the GEV with shape $$\xi$$. For instance the exponential and Gumbel distributions are tail-equivalent. However, the converse is not true: distributions within the same domain of attraction or even with the same tail index are generally not tail-equivalent. For example the normal distribution or the gamma distribution with a shape $$\neq 1$$ are not tail-equivalent to the GP distribution with shape $$\xi = 0$$ i.e. to the exponential distribution. Yet both of these distributions are in the domain of attraction of the GEV distribution with $$\xi = 0$$ i.e. the Gumbel domain of attraction. Within this domain, we find quite different tails, some being "heavier" than the exponential/Gumbel - such as a Gamma with shape $$< 1$$, a Weibull with shape $$>1$$ or a log-normal - and some being "lighter" - such as a Gamma with shape $$> 1$$.

• What is $\omega$ in the above answer? I'm interpreting this to mean a specific $x$. Otherwise, I'd interpret the equation given as indicating that the two tails have the same asymptotic form, but to within a multiplicative constant. Commented Sep 13, 2019 at 13:05
• Yves, given your useful reply, let me ask, then, given a set of distributions in a common domain of attraction, it seems that some of these might be very different ... so much so, that some of them might fit my data a lot better than others (using standard measures of goodness of fit). Can you comment on this? Commented Sep 13, 2019 at 13:30
• This possibility would explain, for example, the confusion I have over the textbook assertions that a GP with zero shape parameter leads to "tails that decrease exponentially, such as the normal". To me, the exponential and the normal have very different tails, but might, under my interpretation of what you've written, be within the same domain of attraction. Commented Sep 13, 2019 at 13:38
• @Isambard Kingdom The $\omega$ is the so-called upper end-point, that is: the upper bound of the support. Yes there are very different distributions in a given DA. In Extreme-Value applications we care about observations that are sampled in a specific way (largest in a time period or those exceeding a high threshold) so the distribution does not matter so much: we are interested in the DA. When the original distribution is very different from the GEV of its DA, the convergence to it can be very slow, see this question.
– Yves
Commented Sep 13, 2019 at 14:42
• Hmm, I would have thought that the specific distribution still matters. Different distributions give different goodness of fits, though I can understand that for just a few extreme data, various measures of goodness of fit might not be especially diagnostic. My concern is about extrapolation, and here different fits can lead to different estimates. Commented Sep 13, 2019 at 15:47