Given the random variable
$$Y = \max(X_1, X_2, \ldots, X_n)$$
where $X_i$ are IID uniform variables, how do I calculate the PDF of $Y$?
$\begingroup$
$\endgroup$
4
-
7$\begingroup$ If this is homework, please read the FAQ and update your question accordingly. $\endgroup$– cardinalCommented Nov 15, 2011 at 20:01
-
1$\begingroup$ Can one use Vandermonde’s identity to show joint function of 2 order Statistics say F_y(r)*G_y(r) ? $\endgroup$– larry mintzCommented Jun 9, 2018 at 21:10
-
1$\begingroup$ Out of interest, what course covers this kind of problem? It is not something that I encountered in my engineering probability course. $\endgroup$– AlexCommented Aug 26, 2019 at 22:27
-
1$\begingroup$ @Alex What about a statistics course that covers resampling? $\endgroup$– SOFeCommented Oct 1, 2019 at 13:38
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
6
It is possible that this question is homework but I felt this classical elementary probability question was still lacking a complete answer after several months, so I'll give one here.
From the problem statement, we want the distribution of
$$Y = \max \{ X_1, ..., X_n \}$$
where $X_1, ..., X_n$ are iid ${\rm Uniform}(a,b)$. We know that $Y < x$ if and only if every element of the sample is less than $x$. Then this, as indicated in @varty's hint, combined with the fact that the $X_i$'s are independent, allows us to deduce
$$ P(Y \leq x) = P(X_1 \leq x, ..., X_n \leq x) = \prod_{i=1}^{n} P(X_i \leq x) = F_{X}(x)^n$$
where $F_{X}(x)$ is the CDF of the uniform distribution that is $\frac{y-a}{b-a}$. Therefore the CDF of $Y$ is $$F_{Y}(y) = P(Y \leq y) = \begin{cases} 0 & y \leq a \\ \phantom{} \left( \frac{y-a}{b-a} \right)^n & y\in(a,b) \\ 1 & y \geq b \\ \end{cases}$$
Since $Y$ has an absolutely continuous distribution we can derive its density by differentiating the CDF. Therefore the density of $Y$ is
$$ p_{Y}(y) = \frac{n(y-a)^{n-1}}{(b-a)^{n}}$$
In the special case where $a=0,b=1$, we have that $p_{Y}(y)=ny^{n-1}$, which is the density of a Beta distribution with $\alpha=n$ and $\beta=1$, since ${\rm Beta}(n,1) = \frac{\Gamma(n+1)}{\Gamma(n)\Gamma(1)}=\frac{n!}{(n-1)!} = n$.
As a note, the sequence you get if you were to sort your sample in increasing order - $X_{(1)}, ..., X_{(n)}$ - are called the order statistics. A generalization of this answer is that all order statistics of a ${\rm Uniform}(0,1)$ distributed sample have a Beta distribution, as noted in @bnaul's answer.
-
$\begingroup$ This actually was a homework question for me. Thanks for the explanation. $\endgroup$– Paul P MCommented Mar 26, 2016 at 22:20
-
$\begingroup$ i feel like i should be able to take your insights here and answer this question, but i'm not seeing how to do that. can you help me out? can you recommend a textbook or chapter that speaks to this general issue? $\endgroup$– user28511Commented Mar 31, 2016 at 20:38
-
2$\begingroup$ @PaulPM Out of interest, what course covers this kind of problem? It is not something that I encountered in my engineering probability course. $\endgroup$– AlexCommented Aug 26, 2019 at 22:27
-
$\begingroup$ @Alex this comes up in MIT's 6.041/6.431, Introduction to Probability (archived on MITx and OCW) $\endgroup$ Commented Apr 7, 2021 at 1:58
-
1$\begingroup$ @julianstanley Hi! Could you specify which lecture of MIT's 6.041/6.431 deals with this concept? I did a quick scan of the class PPTs but could not find them. I might have missed it or something. But I would really appreciate it if you could state the lecture number as I am rushing against a deadline to solve a similar problem. Thanks in advance!! $\endgroup$ Commented Jul 1, 2021 at 6:12
$\begingroup$
$\endgroup$
7
The maximum of a sample is one of the order statistics, in particular the $n$th order statistic of the sample $X_1,\dots,X_n$. In general, computing the distribution of order statistics is difficult, as described by the Wikipedia article; for some special distributions, the order statistics are well-known (e.g. for the uniform distribution, which has Beta-distributed order statistics).
EDIT: The Wikipedia article on sample maximum and minimum is also helpful and more specific to your problem.
-
6$\begingroup$ For distributions with densities, computing the marginal distribution of a particular order statistic is quite straightforward. It is even easier for "special" order statistics like the minimum and maximum. $\endgroup$– cardinalCommented Nov 15, 2011 at 20:03
-
$\begingroup$ I guess it depends on what is meant by "calculate" in the original question. Certainly doing so numerically is straightforward; I interpreted the question as asking how to find a closed form solution, which is in general not easy. $\endgroup$– bnaulCommented Nov 15, 2011 at 20:59
-
9$\begingroup$ @bnaul: Let $F(x) = \mathbb P(X \leq x)$ be an arbitrary distribution function and let $X_1,\ldots,X_n$ be an iid sample from $F$. Let $\newcommand{Xk}{X_{(k)}}\Xk$ be the $k$th order statistic. Then $$\mathbb P(\Xk \leq x) = \sum_{m=k}^n \mathbb P( |\{i: X_i \leq x\}| = m) = \sum_{m=k}^n {n \choose m} F(x)^m (1-F(x))^{n-m} \>.$$ QED. $\endgroup$– cardinalCommented Nov 15, 2011 at 21:16
-
1$\begingroup$ Perhaps a way to understand cardinals answer (given that you understand order statistic for uniform) is that because cdfs are monotonic 1-to-1 transformations of a uniform cdf, we can always express the event {X<a} in terms of a uniform random variable (this is why monte carlo works). So any result based on a uniform distribution will easily generalise to other random variables - just apply the transformation $U=F_{X}(X)$. $\endgroup$ Commented Feb 14, 2012 at 1:01
-
2$\begingroup$ @probabilityislogic: The intuition is good, though it seems you have continuous random variables in mind in your comment. (The result in my second comment above, e.g., works for an arbitrary distribution function.) $\endgroup$– cardinalCommented Feb 21, 2012 at 13:04
$\begingroup$
$\endgroup$
If $F_{Y}(y)$ is the CDF of $Y$, then $$F_Y(y)=\text{Prob}(y>X_1,y>X_2,...,y>X_n)$$ You can then use the iid property and the cdf of a uniform variate to compute $F_Y(y)$.
$\begingroup$
$\endgroup$
2
The maximum of a set of IID random variables when appropriately normalized will generally converge to one of the three extreme value types. This is Gnedenko's theorem,the equivalence of the central limit theorem for extremes. The particular type depends on the tail behavior of the population distribution. Knowing this you can use the limiting distribution to approximate the distribution for the maximum.
Since the uniform distribution on [a, b] is the subject of this question Macro has given the exact distribution for any n and a very nice answer. The result is rather trivial. For the normal distribution a nice closed form is not possible but appropriately normalized the maximum for the normal converges to the Gumbel distribution F(x)=exp(- e $^-$$^x$).
For the uniform the normalization is (b-a)-x/n and F$^n$(b-a-x/n)=(1-x/[n(b-a)])$^n$
which converges to e$^-$$^x$$^/$$^($$^b$$^-$$^a$$^)$. Note here that y=b-a-x/n. and F$^n$(y) converges to 1 as y goes to b-a. This holds for all 0
In this case it is easy to compare the exact value to its asymptotic limit.
answered Jul 20, 2012 at 0:59
-
5$\begingroup$ For this answer to be practicable, you need to stipulate, in detail, how one "appropriately normalizes" the values and you also need to provide some way to estimate how large $n$ must be before the asymptotic formula becomes a reliable approximation. $\endgroup$– whuber ♦Commented Jul 20, 2012 at 15:27
-
$\begingroup$ @whuber Anyone can look at Gnedenko's theorem to see the normalization. Equally as important is the tail characteristics that determine which of the three types applies. The theorem generalizes to stationary stochastic processes. So anyone who want to know the nitty gritty details can look at Leadbetter's book or my PhD thesis. When n is large enough is a difficult question to answer for any form of asymptotics. I guess the Berry-Esseen theorem helps for the central limit theorem. I don't know what is comparable for extremes. $\endgroup$ Commented Jul 20, 2012 at 16:57