# Normalization to non-degenerate distribution

I am reading de Haan's Extreme Value Theory (2006). In the discussion of distribution of sample maximum, he said "in order to obtain a non-degenerate limit distribution, a normalization is necessary". Then he gave the following example. "Suppose that there exists a sequence of constants $(a_n)>0$ and $(b_n)$ such that

$$\frac{\max \{X_1, \cdots, X_n\} - b_n}{a_n} (1)$$

has a non-degenerate limit distribution as $n \to \infty$, i.e., $$\lim_n F^n(a_nx + b_n)=G(x), (2)$$ for every continuity point $x$ of $G$, where $G$ is a non-degenerate distribution function." And he also commented that this is a linear normalization.

I have three questions here.

1. What does it mean to normalize to a non-degenerate distribution function, please? In my past studies, normalization means to find the constant $c=\frac{1}{\sqrt{2\pi}}$ such that $\int_\mathbb R c e^{-\frac{x^2}{2}} = 1$. It appears that normalization means different things in de Haan's book.
2. What do the two sequences $(a_n)$ and $(b_n)$ mean here, please? Or what role do they play, please? Why is $(1)$ equivalent to $(2)$, please?
3. What are common non-linear normalization, please? Thank you!
-
"What does it mean to normalized a degenerate distribution function, please?" -- where in the text you quoted is anyone attempting to do that? Please highlight the part where that is. All I see is discussion of normalization to achieve non-degenerate G – Glen_b Aug 17 '14 at 8:21
"All I see is discussion of normalization to achieve non-degenerate G." What do you mean by "normalization to achieve non-degenerate G"? I gave an example on normalization which I know of in my question. But I suppose there are other meaning attached to this word. – LaTeXFan Aug 17 '14 at 8:26
Can you edit your question to reflect this change from degenerate? – Glen_b Aug 17 '14 at 9:28
@Glen_b I do not know what you mean. Where do I need to edit, please? – LaTeXFan Aug 17 '14 at 9:44
I've changed it. See the edit history for what was altered. – Glen_b Aug 17 '14 at 11:54

Consider the most basic example, the sample mean from an i.i.d. sample of size $n$, $\bar X_n$.

We know that as $n \rightarrow \infty$, $\bar X_n \rightarrow \mu$, where $\mu$ is the common mean, the expected value, of the random variables from which the sample is generated.

So at the limit, $\bar X$ has a degenerate distribution, which is the formal way to say that it convergences to a constant. Constant terms can be considered as degenerate random variables. We usually say "constants do not have a distribution", but since sometimes issues of existence matter (meaning that the phrase "the distribution does not exist" properly means that the statistic we examine goes to infinity as the sample size goes to infinity), the correct way to distinguish the two cases is to say "the distribution of a constant is degenerate".

And what do we do, in order to obtain a non-degenerate asymptotic distribution? We create a function of the sample mean, that does not converge to a constant, but it doesn't diverge either. In the case of the sample mean, this function is $\sqrt n(\bar X_n -\mu)$.

In analogous spirit, in Extreme Value Theory, the extreme order statistics, either diverge (if the distribution has unbounded support), or tend to a constant (if the distribution has bounded support on their side). In both cases, we don't get a limiting distribution. So we need to find a function of the extreme order statistic, which will converge to a non-constant random variable and hence, with a useable distribution. The deterministic series $\{a_n\}$ and $\{b_n\}$, together with the statistic, create this function. Finding these series is not that simple, see for example this post.

Regarding the example given by @Glen_b for the maximum order statistic from a Uniform $U(0,1)$, a distribution with bounded support, intuitively, as the sample size increases, we will obtain at least one realization of the random variable that exactly equals its upper bound. But this means that $X_{(n)} \rightarrow \max X$, which is a constant, and so it has a degenerate distribution. So we need to find a function of $X_{(n)}$ that does not diverge, and does converge to a random variable. In the specific case, this function is indeed $Z = n(1-X_{(n)})$. To see this, use the change of variable formula to find that

$$Z =n(1-X_{(n)}) \Rightarrow X_{(n)} = 1-\frac Zn \Rightarrow \left|\frac {\partial X}{\partial Z} \right|= \frac 1n$$

and note that $Z \in [0,n]$. Therefore

$$f_Z(z) = \left|\frac {\partial X}{\partial Z} \right| f_{X_{(n)}}(1-z/n) = \frac 1n \left (nf_X(1-z/n)[F_X(1-z/n)]^{n-1}\right)$$

But $f_X(\cdot) =1$, and $F_X(x) =x$. So

$$f_Z(z) =\left(1-\frac zn\right)^{n-1}$$

and

$$F_Z(z) = \int_{0}^z\left(1-\frac tn\right)^{n-1}dt = 1-\left(1-\frac zn\right)^{n}$$

Then

$$\lim_{n\rightarrow \infty}F_Z(z) = 1-\lim_{n\rightarrow \infty}\left(1-\frac zn\right)^{n} = 1-e^{-z}$$

which is the distribution function of a standard exponential (i.e. with mean value $1$).

-
Thank you very much for such detailed answers. It all sounds cool and fun. However, why do you we want to do this kind of things in the first place. In normal case, the sample mean is also normal with parameters $\mu$ and $\sigma^2/n$. Isn't this enough, please? – LaTeXFan Aug 18 '14 at 7:45
Enough? 1) There are myriads of real-life phenomena that cannot be modeled using the normal distribution. 2) What does the sample maximum has to do with the sample mean? 3) But even in the case you mention, the variance $\sigma^2/n$ becomes negligibly small as the sample size increases... and in such a case, without "all this kind of stuff", Statistics would be able to provide us essentially only point estimates of the things we don't know, which in many cases is the least we are interested in. – Alecos Papadopoulos Aug 18 '14 at 10:03
Enlightening. Thanks. – LaTeXFan Aug 18 '14 at 10:06

Normalization is used to mean a variety of things - which usually relate to scaling in some way. In this case it's just a matter of finding constants to subtract and divide by such that the resulting sequence of random variables converges to a distribution that isn't degenerate.

Presumably in the situation under discussion,

$$\max \{X_1, \cdots, X_n\}$$

is degenerate (it's typically the case).

Aside from some oddness in that they seem to be using one letter for two different things there, all they're talking about is choosing $a_n$ and $b_n$ so that

$$\frac{\max \{X_1, \cdots, X_n\} - b_n}{a_n}$$

isn't degenerate in the limit.

If you can find $E(\max \{X_1, \cdots, X_n\})$ and $\text{Var}(\max \{X_1, \cdots, X_n\})$ as functions of $n$, for example, you might be able to set $b_n$ to the first and $a_n$ to the square root of the second, which would yield something that has constant mean and variance ($0$ and $1$ respectively). If the distribution converges in the limit, it should satisfy the conditions.

For example, consider $X_i$ being U(0,1). Then in the limit, the sample maximum $X_{(n)}$ is degenerate.

But I think $n(1-X_{(n)})$ is not degenerate in the limit - IIRC it goes to a standard exponential.

-
Thank you. Why is $X_{(n)}$ degenerate, please? And what is IIRC? – LaTeXFan Aug 17 '14 at 9:47
In the example I gave, consider the variance of $X_{(n)}$ in the limit (it's 0). In other cases, it's the mean you need to worry about. – Glen_b Aug 17 '14 at 11:40