45
$\begingroup$

Given the random variable

$$Y = \max(X_1, X_2, \ldots, X_n)$$

where $X_i$ are IID uniform variables, how do I calculate the PDF of $Y$?

$\endgroup$
  • 4
    $\begingroup$ If this is homework, please read the FAQ and update your question accordingly. $\endgroup$ – cardinal Nov 15 '11 at 20:01
  • $\begingroup$ Can one use Vandermonde’s identity to show joint function of 2 order Statistics say F_y(r)*G_y(r) ? $\endgroup$ – larry mintz Jun 9 '18 at 21:10
62
$\begingroup$

It is possible that this question is homework but I felt this classical elementary probability question was still lacking a complete answer after several months, so I'll give one here.

From the problem statement, we want the distribution of

$$Y = \max \{ X_1, ..., X_n \}$$

where $X_1, ..., X_n$ are iid ${\rm Uniform}(a,b)$. We know that $Y < x$ if and only if every element of the sample is less than $x$. Then this, as indicated in @varty's hint, combined with the fact that the $X_i$'s are independent, allows us to deduce

$$ P(Y \leq x) = P(X_1 \leq x, ..., X_n \leq x) = \prod_{i=1}^{n} P(X_i \leq x) = F_{X}(x)^n$$

where $F_{X}(x)$ is the CDF of the uniform distribution. Therefore the CDF of $Y$ is $$F_{Y}(y) = P(Y \leq y) = \begin{cases} 0 & y \leq a \\ \phantom{} \left[ (y-a)/(b-a) \right]^n & y\in(a,b) \\ 1 & y \geq b \\ \end{cases}$$

Since $Y$ has an absolutely continuous distribution we can derive its density by differentiating the CDF. Therefore the density of $Y$ is

$$ p_{Y}(y) = \frac{n(y-a)^{n-1}}{(b-a)^{n}}$$

In the special case where $a=0,b=1$, we have that $p_{Y}(y)=ny^{n-1}$, which is the density of a Beta distribution with $\alpha=n$ and $\beta=1$, since ${\rm Beta}(n,1) = \frac{\Gamma(n+1)}{\Gamma(n)\Gamma(1)}=\frac{n!}{(n-1)!} = n$.

As a note, the sequence you get if you were to sort your sample in increasing order - $X_{(1)}, ..., X_{(n)}$ - are called the order statistics. A generalization of this answer is that all order statistics of a ${\rm Uniform}(0,1)$ distributed sample have a Beta distribution, as noted in @bnaul's answer.

$\endgroup$
  • $\begingroup$ This actually was a homework question for me. Thanks for the explanation. $\endgroup$ – Paul P M Mar 26 '16 at 22:20
  • $\begingroup$ i feel like i should be able to take your insights here and answer this question, but i'm not seeing how to do that. can you help me out? can you recommend a textbook or chapter that speaks to this general issue? $\endgroup$ – user28511 Mar 31 '16 at 20:38
5
$\begingroup$

The maximum of a sample is one of the order statistics, in particular the $n$th order statistic of the sample $X_1,\dots,X_n$. In general, computing the distribution of order statistics is difficult, as described by the Wikipedia article; for some special distributions, the order statistics are well-known (e.g. for the uniform distribution, which has Beta-distributed order statistics).

EDIT: The Wikipedia article on sample maximum and minimum is also helpful and more specific to your problem.

$\endgroup$
  • 5
    $\begingroup$ For distributions with densities, computing the marginal distribution of a particular order statistic is quite straightforward. It is even easier for "special" order statistics like the minimum and maximum. $\endgroup$ – cardinal Nov 15 '11 at 20:03
  • $\begingroup$ I guess it depends on what is meant by "calculate" in the original question. Certainly doing so numerically is straightforward; I interpreted the question as asking how to find a closed form solution, which is in general not easy. $\endgroup$ – bnaul Nov 15 '11 at 20:59
  • 8
    $\begingroup$ @bnaul: Let $F(x) = \mathbb P(X \leq x)$ be an arbitrary distribution function and let $X_1,\ldots,X_n$ be an iid sample from $F$. Let $\newcommand{Xk}{X_{(k)}}\Xk$ be the $k$th order statistic. Then $$\mathbb P(\Xk \leq x) = \sum_{m=k}^n \mathbb P( |\{i: X_i \leq x\}| = m) = \sum_{m=k}^n {n \choose m} F(x)^m (1-F(x))^{n-m} \>.$$ QED. $\endgroup$ – cardinal Nov 15 '11 at 21:16
  • 1
    $\begingroup$ Perhaps a way to understand cardinals answer (given that you understand order statistic for uniform) is that because cdfs are monotonic 1-to-1 transformations of a uniform cdf, we can always express the event {X<a} in terms of a uniform random variable (this is why monte carlo works). So any result based on a uniform distribution will easily generalise to other random variables - just apply the transformation $U=F_{X}(X)$. $\endgroup$ – probabilityislogic Feb 14 '12 at 1:01
  • 2
    $\begingroup$ @probabilityislogic: The intuition is good, though it seems you have continuous random variables in mind in your comment. (The result in my second comment above, e.g., works for an arbitrary distribution function.) $\endgroup$ – cardinal Feb 21 '12 at 13:04
1
$\begingroup$

If $F_{Y}(y)$ is the CDF of $Y$, then $$F_Y(y)=\text{Prob}(y>X_1,y>X_2,...,y>X_n)$$ You can then use the iid property and the cdf of a uniform variate to compute $F_Y(y)$.

$\endgroup$
-3
$\begingroup$

The maximum of a set of IID random variables when appropriately normalized will generally converge to one of the three extreme value types. This is Gnedenko's theorem,the equivalence of the central limit theorem for extremes. The particular type depends on the tail behavior of the population distribution. Knowing this you can use the limiting distribution to approximate the distribution for the maximum.

Since the uniform distribution on [a, b] is the subject of this question Macro has given the exact distribution for any n and a very nice answer. The result is rather trivial. For the normal distribution a nice closed form is not possible but appropriately normalized the maximum for the normal converges to the Gumbel distribution F(x)=exp(- e $^-$$^x$).

For the uniform the normalization is (b-a)-x/n and F$^n$(b-a-x/n)=(1-x/[n(b-a)])$^n$

which converges to e$^-$$^x$$^/$$^($$^b$$^-$$^a$$^)$. Note here that y=b-a-x/n. and F$^n$(y) converges to 1 as y goes to b-a. This holds for all 0

In this case it is easy to compare the exact value to its asymptotic limit.

Gumbel's book

Galambos' book

Leadbetter's book

Novak's book

Coles book

$\endgroup$
  • 4
    $\begingroup$ For this answer to be practicable, you need to stipulate, in detail, how one "appropriately normalizes" the values and you also need to provide some way to estimate how large $n$ must be before the asymptotic formula becomes a reliable approximation. $\endgroup$ – whuber Jul 20 '12 at 15:27
  • $\begingroup$ @whuber Anyone can look at Gnedenko's theorem to see the normalization. Equally as important is the tail characteristics that determine which of the three types applies. The theorem generalizes to stationary stochastic processes. So anyone who want to know the nitty gritty details can look at Leadbetter's book or my PhD thesis. When n is large enough is a difficult question to answer for any form of asymptotics. I guess the Berry-Esseen theorem helps for the central limit theorem. I don't know what is comparable for extremes. $\endgroup$ – Michael Chernick Jul 20 '12 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.