# Limiting distribution of the first order statistic of a general distribution

Let $Z_i,Z_2,\ldots$ be IID Random Variables with density $f$. Suppose that $P(Z_i>0)=1$ and that $\lambda=\lim_{x \to 0+} f(x)>0$. How can I show that $X_n=n \times \min\{Z_i\}$ has a limiting exponential distribution with mean$1/\lambda$?

I know that the CDF of the first order Statistic is for $t>0$

$$F_{min \{ Z_i \}} (t)= 1- \left[ 1-F(t) \right]^{n}$$

Convergence in distribution requires that $F_n (t) \to F(t)$ for some distribution function $F$ but I cannot quite take the limit of the above, with $t$ replaced by $x/n$, as the distribution is not known. I guess I have to insert the second piece of information somewhere but I do not recognise where.

Could you please give me a hint or two?

Thank you.

-
One approach is to apply the Fisher-Tippett-Gnedenko theorem to $-Z_i$. – whuber Jun 9 '14 at 17:05
@whuber Thank you, I didn't even know that such thing existed. I wonder though whether a simpler way exists as my knowledge does not stretch that far yet. – JohnK Jun 9 '14 at 17:08
I believe a study of that theorem will pay off here, if only to suggest that what you are trying to prove might not be true--or at least requires some strong additional assumptions about $F$. – whuber Jun 9 '14 at 17:14
Thank you for the reference: in reading it I see I overlooked the assumption that the limiting value of the density at zero is strictly positive: that's a crucial assumption. Intuitively, it tells you that the distribution of the minimum of a very large number of independent variables will be controlled by the value of $f$ near $0$. So one way to appreciate this problem would be to replace $F$ by, say, a uniform distribution, which would have to be U$(0,1/\lambda)$. You can compute the distribution of $X_n$ exactly in this case: what is it and what is its limit as $n\to\infty$? – whuber Jun 9 '14 at 19:13
Thank you @AlecosPapadopoulos. You can certainly post an answer, if you like so I can mark it as the correct one. My sole objection is that the limit we are taking in this case is the limit of a pdf and I do not think we are supposed to do that,in fact I have seen many counterexamples of this practice. Are we not supposed to examine CDF's only? – JohnK Jun 9 '14 at 20:15

(The answer has been reworked to respond to OP's and whuber's comments).

The complementary cdf of $X$ is

$$G_n(x) = \left[1-F_Z\left(x/n\right)\right]^{n}$$

To prove that asymptotically $X$ follows an exponential distribution, we need to show that $$\lim_{n\rightarrow \infty}G_n(x)= e^{-\lambda x}$$

Consider

$$F_Z\left(x/n\right) = \int_0^{x/n}f(t)dt$$

By the properties of the integral, we have

$$\int_0^{x/n}f(t)dt = \frac 1n\int_0^{x}f(t/n)dt$$

Define $$h_n(w) = \left(1+\frac {w}{n}\right)^{n}, \qquad \lim_{n\rightarrow \infty}h_n(w) = e^w=h(w), \;\; w \in \mathbb R$$

and

$$g_n(x) = -\int_0^{x}f(t/n)dt,\;\;\; -\lim_{n\rightarrow \infty}g_n(x) = -\int_0^{x}f(0)dt = -\lambda x = g(x), \;\;x \in \mathbb R_+$$

(To respond to a question by the OP, we can take the limit inside the integral. First note that $n\geq 1$, and we do not send $x$ to infinity. So the argument of $f$ does not explode. So even if it were the case that $f(\infty) \rightarrow \infty$, we do not need to consider this case here. Then, since also $f(0)$ is finite by assumption, $f$ is bounded and dominated convergence holds).

With these definitions we can write

$$G_n(x) = h_n(g_n(x))$$

and the question is

$$\lim_{n\rightarrow \infty}h_n(g_n(x)) =?\;\; h(g(x)) = e^{-\lambda x},\;\;x \in \mathbb R_+$$

The limit of a composition of functions does not in general equal the composition of their limits (which is what whuber has essentially pointed out in his comment). But this equality will hold if $(i)$ the limit of $h_n$ is a continuous function (it is), $(ii)$ the functions $g_n(x)$ map $\mathbb R_+$ to $\mathbb R$ (they do), and $(iii)$ $h_n$ converges uniformly to $h$. This too holds (convergence to $e^w$ is uniform). So the above equality holds and we have proven what we needed to prove.

-
Beautiful, thank you. Are we in the clear about moving the limit inside the integral? – JohnK Jun 9 '14 at 21:17
There is a subtle issue that seems not to be justified in this demonstration: when taking the limits, the same $n$ is involved in the limit of the integrals and the limit of the powers (the antepenultimate and penultimate lines, respectively): without specific analysis, you cannot take those limits separately. (It was the consideration of this point that originally caused me to present a slightly longer answer.) – whuber Jun 9 '14 at 21:27
@whuber whenever you have some time, I would appreciate if you could have a look at my re-worked answer - I tried to respond to your comment. – Alecos Papadopoulos Jun 10 '14 at 0:22
You have definitely answered my question, thank you. – JohnK Jun 10 '14 at 7:54
I believe you can seriously consider amazon.com/… – Alecos Papadopoulos Jun 11 '14 at 22:27

To prove convergence in distribution we need to show that the complementary distribution of $X_n$, written $G_n$ where $G_n(x)=\Pr(X_n\gt x)$, gets close to an exponential function for $n$ sufficiently large. To this end, let $t\gt 0$ be an arbitrary point at which to evaluate $G_n(t)$. Note that the independence of the $Z_i$ implies

$$G_n(t) = \left(1 - F\left(\frac{t}{n}\right)\right)^n = \left(1 - \lambda\frac{t}{n} + \left[\lambda\frac{t}{n} - F\left(\frac{t}{n}\right)\right]\right)^n.$$

The term in square brackets is the problem--if it weren't there the limit would obviously be exponential--so we will use the only information available to us to estimate it and hope that it's very small for large $n$. The existence of the limit

$$\lambda = {\lim}_{x\to 0^{+}} f\left(x\right)$$

implies

$$\left|\lambda\frac{t}{n} - F\left(\frac{t}{n}\right)\right| = \left|\int_0^{t/n} (\lambda - f(u)) du\right| \le \frac{t}{n}\sup_{0\le u\le t/n}\left(|\lambda - f(u)|\right) = \frac{t}{n}\varepsilon(n)$$

for some function $\varepsilon$ that approaches $0$ for large arguments. Substitute this into the foregoing and assume $n$ is so large that $F\left(\frac{t}{n}\right)\lt 1$, so that we may take logarithms, and use the Taylor series of the logarithm near $1$ to estimate

\eqalign{ \log(G(t))=n\log\left(1 - F\left(\frac{t}{n}\right)\right) &= n\log\left(1 - \lambda\frac{t}{n} + \left[\lambda\frac{t}{n} - F\left(\frac{t}{n}\right)\right]\right) \\ &= n\log\left(1 - \left(\lambda-\varepsilon(n)\right)\frac{t}{n}\right) \\ &= -\left(\lambda-\varepsilon(n)\right)t + \left[(\lambda - \varepsilon(n))t\right]^2O\left(\frac{1}{n}\right). }

Clearly (applying theorems about the limits of products and sums of continuous functions) this has a limit as $n\to \infty$ and it equals $-\lambda t$, showing that $G(t)=\exp(\log(G(t))$ has the limiting value $\exp(-\lambda t)$, QED.

-
Referring to the question's comment thread, when $n$ is very large so that $t/n$ is very small (but positive), we can characterize $\lambda\frac{t}{n} - F\left(\frac{t}{n}\right)$ as the discrepancy between $F$ and a Uniform$(\lambda)$ distribution near $0$. Thus this argument is simply a semi-rigorous restatement of the intuition expressed in those comments. (For even more rigor one would write a full epsilon-delta limiting argument, but it should now be obvious how that proof would go.) – whuber Jun 9 '14 at 20:58