I am trying to understand how to get from the CDF to the PDF of the kth order statistic and I am following this article.

enter image description here

I understand that I have to take the derivative of F to get f. I also understand the individual parts of f that are explained later in the article. In general I know that I should be able to differentiate all parts of the sum separately, but probably the binomial coefficient confuses me and why the sum disappears ("the derivative of the sum of two functions is just the sum of their derivatives").

And what is the difference between f_k(x) and f_x(x)? Currently the PDF looks kind of recursive to me.

Disclaimer: this if for a homework example, so I don't want a step by step solution, but maybe some helpful hints.

  • $\begingroup$ Technically, what you listed are correct only if $F$ is continuous. The formula needs to be modified (to a more complicated form) if $F$ is discrete. See this question to get more insight. $\endgroup$
    – Zhanxiong
    Jan 20, 2023 at 22:38

2 Answers 2


The $r$-th order statistic $X_{(r)}$ has cdf $$F_{X_{(r)}}(x) = \sum_{j=r}^{n} \binom nj [ F_{X}(x) ]^{j} [ 1 - F_{X}(x) ]^{n-j}$$ because \begin{align}\mathbb P(X_{(r)}\le x) &= \mathbb P(\exists~i_1,\ldots,i_r,~X_{i_1}\le x,\ldots,X_{i_r}\le x)\\ &=\mathbb P(\exists~i_1,\ldots,i_r,~X_{i_1}\le x,\ldots,X_{i_r}\le x,x<X_{i_{r+1},\ldots})\\&+\mathbb P(\exists~i_1,\ldots,i_{r+1},~X_{i_1}\le x,\ldots,X_{i_{r+1}}\le x,x<X_{i_{r+2},\ldots})+\\&\qquad\qquad\vdots\\ &+\mathbb P(X_1\le x,\cdots,X_n\le x)\end{align} The pdf is obtained by derivation of in $x$: \begin{align}f_{X_{(r)}}(x) &= \frac{\text d}{\text dx}F_{X_{(r)}}(x) \\&= \frac{\text d}{\text dx} \sum_{j=r}^{n} \binom nj [ F_{X}(x) ]^{j} [ 1 - F_{X}(x) ]^{n-j} \\&= \sum_{j=r}^{n} \binom nj \frac{\text d}{\text dx} [ F_{X}(x) ]^{j} [ 1 - F_{X}(x) ]^{n-j} \\&= \cdots \\&= \dfrac{(n-1)!}{(r-1)!{(n-r)}!}~f_{X}(x) [ F_{X}(x) ]^{r-1} [ 1 - F_{X}(x) ]^{n-r}\end{align} with the sum disappearing due to Newton's binomial theorem.

The notations indicate the distinction between the distributions of $X_i$, an arbitrary term in the iid sample, and $X_{(i)}$, the $i$-th order statistic.

  • $\begingroup$ Xi'an, I have added a relevant parenthesis. Also, would it be $(n-1)! $ in the numerator or $n! $? +1 for the step by step derivation. $\endgroup$ Jan 20, 2023 at 19:31

I think the derivative part needs more details: \begin{equation} \begin{aligned} \frac{d}{dx} F_{X_{(r)}}(x) &= \frac{d}{dx}\left\{\sum_{j=r}^nC^n_j \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j}\right\}\\ &=\frac{d}{dx}\left\{\sum_{j=r}^{n-1} C^n_j \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j} + \left[F_X(x)\right]^n\right\} \end{aligned} \end{equation} Using derivative by parts, we have \begin{equation} \begin{aligned} \frac{d}{dx} F_{X_{(r)}}(x) =&~ \sum_{j=r}^{n-1} C^n_j \times jf_X(x) \left[F_X(x)\right]^{j-1}\left[1 - F_X(x)\right]^{n-j} \\ &- \sum_{j=r}^{n-1} C^n_j \times (n-j)f_X(x) \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j-1} + nf_X(x)\left[F_X(x)\right]^{n-1}\\ = &~C^n_r r f_X(x) \left[F_X(x)\right]^{r-1}\left[1 - F_X(x)\right]^{n-r} + \sum_{j=r+1}^{n-1} C^n_j jf_X(x) \left[F_X(x)\right]^{j-1}\left[1 - F_X(x)\right]^{n-j} \\ &- \sum_{j=r}^{n-2} C^n_j (n-j)f_X(x) \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j-1} - nf_X(x)\left[F_X(x)\right]^{n-1} + nf_X(x)\left[F_X(x)\right]^{n-1}\\ =&~ \frac{n!}{(r-1)!(n-r)!}f_X(x) \left[F_X(x)\right]^{r-1}\left[1 - F_X(x)\right]^{n-r} \\ & + \sum_{j=r+1}^{n-1} \frac{n!}{(j-1)!(n-j)!}f_X(x) \left[F_X(x)\right]^{j-1}\left[1 - F_X(x)\right]^{n-j} \\ &- \sum_{j=r}^{n-2} \frac{n!}{j![n-(j+1)]!}f_X(x) \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j-1}\\ =&~\frac{n!}{(r-1)!(n-r)!}f_X(x) \left[F_X(x)\right]^{r-1}\left[1 - F_X(x)\right]^{n-r} \end{aligned} \end{equation}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.