Determine the limiting distribution of Standard Normal order statistics

Question

Let $X_1,...,X_n$ be an i.i.d sample from the standard normal distribution. Is there any general formula for the first order statistic of this sample? For example, I have seen a formula by Blom (1958): $e(r:n) \approx \mu + \Phi^{-1}(\frac{r-\alpha}{n-2\alpha+1})\sigma$. My ultimate goal is actually finding the limiting distribution of the first order statistics for this standard normal sample, will the formula be any helpful? Any suggestion on finding the limiting distribution?

Answer 1

Let $X_1, \dots,X_n, \dots \sim \textrm{ iid } \mathcal N(0,1)$ and let $Y_n = \min\{X_1, \dots, X_n\}$.

The standard way to study iid minima is via the following computation: $$ P(Y_n \leq y) = 1 - P(Y_n \geq y) = 1-P(X_1 \geq y \cap \dots \cap X_n\geq y) $$ $$ = 1-\left(1-\Phi(y)\right)^n. $$

If you are interested in the distribution of $Y_n$ as $n \to \infty$ then we have $$ \lim_{n \to \infty} P(Y_n \leq y) = \lim_{n \to \infty} 1 - \left(1 - \Phi(y)\right)^n = 1 $$ for any $y \in \mathbb R$. This means the limiting distribution is effectively a point mass at $-\infty$. This makes sense because any interval $(-\infty, a)$ has positive probability for a standard normal RV, so as you sample more and more eventually you'll land in $(-\infty, a)$ for any $a$, so the minimum (in the limit) with probability 1 is less than $a$.

Your formula agrees with this (assuming $r$ specifies which order statistic we care about, so in this case $r=1$): $$ \lim_{n \to \infty} \mu + \Phi^{-1}\left(\frac{1-\alpha}{n-2\alpha+1}\right)\sigma = -\infty $$ although I wouldn't trust this as an argument in its own right without careful analysis since without knowing more details about the approximation it's not safe to assume it holds or is meaningful in the limit.

Answer 2

You can do quite a lot with the distribution of minimum of $n$ iid standard normals, numerically and graphically. For now I will give some R functions. Much background information we do not need to repeat here can be found in the answer here: Approximate order statistics for normal random variables

The density function of minimum of $n$ iid (independent and identically distributed) standard normal random variables:

dmin_norm  <-  function(x, mean=0, sd=1, n, log=FALSE) {
    logdens  <-  log(n)+(n-1)*pnorm(x, mean=mean, sd=sd, lower.tail=FALSE, log.p=TRUE)+dnorm(x, mean=mean, sd=sd, log=TRUE)
    retval  <-  if (log) logdens else exp(logdens)
    retval
    }

Let us first check numerically that this is a density function:

integrate(function(x) dmin_norm(x, n=50), lower=-Inf, upper=Inf)
1 with absolute error < 8.4e-05

We can also find the expectation numerically:

integrate(function(x) x*dmin_norm(x, n=10), lower=-Inf, upper=Inf)
-1.538753 with absolute error < 1.3e-06

It could also be useful to have the quantile function:

qmin_norm  <-  function(q, mean=0, sd=1, n, log.p=FALSE) {
    retval <- qnorm(1-(1-q)^(1/n), mean=mean, sd=sd, log.p=log.p)
    retval
    }

Then I will show a plot of the densities for 5 selected values of n:

Then the asymptotic theory of minima (and maxima). Those two theories are parallel, so I will discuss the maximum in the continuation. I will not give details, mainly a few results and illustrations with R code and graphs. The theory can be found in chapter 8 of "A First Course in Order Statistics" by Arnold, Balakrishnan and Nagaraja.

One starts by asking if there exist some sequences of constants (which not will be unique) $a_n$, $b_n$ such that $$ \frac{X_{(n)}-a_n}{b_n} $$ converges in distribution ($n \rightarrow \infty$) to a limiting random variable $W$. It turns out that when such sequences exists, then the limiting distribution $W$ is of one of three types only. But there do exist examples where no convergence is possible! In the case of interest now, when the parent distribution is normal, it turns out that the limiting distribution is $$ G_3(x) = \exp\left(-\exp(-x)\right) $$ and the constants can be taken as $$ a_n =\sqrt{2\log n} - \frac12 \frac{\log(4\pi\log n)}{\sqrt{2\log n}}, \\ b_n = \frac1{\sqrt{2\log n}} $$ Using the exact distribution of the maximum $F_{(n)}$, found as above, the exact distribution of the transformed maximum is $$ F_{(n)}(a_n + b_n y) $$ which is then approximated by the distribution function $G_3$ given above. A plot showing the exact and approximate distribution for some values of $n$ is

Observe that in the upper tail the approximation seems to be quite good, not so in the lower tail. What is more, it do not seem to improve much with increasing $n$! The cited book says that for "intermediate $n$" one of the other limiting distributions, denoted $G_2$ (not given here), tends to be better, before, eventually, the theoretical limit $G_3$ overtakes.