Deriving the PDF of the kth order statistic from the CDF

Question

I am trying to understand how to get from the CDF to the PDF of the kth order statistic and I am following this article.

I understand that I have to take the derivative of F to get f. I also understand the individual parts of f that are explained later in the article. In general I know that I should be able to differentiate all parts of the sum separately, but probably the binomial coefficient confuses me and why the sum disappears ("the derivative of the sum of two functions is just the sum of their derivatives").

And what is the difference between f_k(x) and f_x(x)? Currently the PDF looks kind of recursive to me.

Disclaimer: this if for a homework example, so I don't want a step by step solution, but maybe some helpful hints.

Answer 1

The $r$-th order statistic $X_{(r)}$ has cdf $$F_{X_{(r)}}(x) = \sum_{j=r}^{n} \binom nj [ F_{X}(x) ]^{j} [ 1 - F_{X}(x) ]^{n-j}$$ because \begin{align}\mathbb P(X_{(r)}\le x) &= \mathbb P(\exists~i_1,\ldots,i_r,~X_{i_1}\le x,\ldots,X_{i_r}\le x)\\ &=\mathbb P(\exists~i_1,\ldots,i_r,~X_{i_1}\le x,\ldots,X_{i_r}\le x,x<X_{i_{r+1},\ldots})\\&+\mathbb P(\exists~i_1,\ldots,i_{r+1},~X_{i_1}\le x,\ldots,X_{i_{r+1}}\le x,x<X_{i_{r+2},\ldots})+\\&\qquad\qquad\vdots\\ &+\mathbb P(X_1\le x,\cdots,X_n\le x)\end{align} The pdf is obtained by derivation of in $x$: \begin{align}f_{X_{(r)}}(x) &= \frac{\text d}{\text dx}F_{X_{(r)}}(x) \\&= \frac{\text d}{\text dx} \sum_{j=r}^{n} \binom nj [ F_{X}(x) ]^{j} [ 1 - F_{X}(x) ]^{n-j} \\&= \sum_{j=r}^{n} \binom nj \frac{\text d}{\text dx} [ F_{X}(x) ]^{j} [ 1 - F_{X}(x) ]^{n-j} \\&= \cdots \\&= \dfrac{(n-1)!}{(r-1)!{(n-r)}!}~f_{X}(x) [ F_{X}(x) ]^{r-1} [ 1 - F_{X}(x) ]^{n-r}\end{align} with the sum disappearing due to Newton's binomial theorem.

The notations indicate the distinction between the distributions of $X_i$, an arbitrary term in the iid sample, and $X_{(i)}$, the $i$-th order statistic.

Answer 2

I think the derivative part needs more details: \begin{equation} \begin{aligned} \frac{d}{dx} F_{X_{(r)}}(x) &= \frac{d}{dx}\left\{\sum_{j=r}^nC^n_j \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j}\right\}\\ &=\frac{d}{dx}\left\{\sum_{j=r}^{n-1} C^n_j \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j} + \left[F_X(x)\right]^n\right\} \end{aligned} \end{equation} Using derivative by parts, we have \begin{equation} \begin{aligned} \frac{d}{dx} F_{X_{(r)}}(x) =&~ \sum_{j=r}^{n-1} C^n_j \times jf_X(x) \left[F_X(x)\right]^{j-1}\left[1 - F_X(x)\right]^{n-j} \\ &- \sum_{j=r}^{n-1} C^n_j \times (n-j)f_X(x) \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j-1} + nf_X(x)\left[F_X(x)\right]^{n-1}\\ = &~C^n_r r f_X(x) \left[F_X(x)\right]^{r-1}\left[1 - F_X(x)\right]^{n-r} + \sum_{j=r+1}^{n-1} C^n_j jf_X(x) \left[F_X(x)\right]^{j-1}\left[1 - F_X(x)\right]^{n-j} \\ &- \sum_{j=r}^{n-2} C^n_j (n-j)f_X(x) \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j-1} - nf_X(x)\left[F_X(x)\right]^{n-1} + nf_X(x)\left[F_X(x)\right]^{n-1}\\ =&~ \frac{n!}{(r-1)!(n-r)!}f_X(x) \left[F_X(x)\right]^{r-1}\left[1 - F_X(x)\right]^{n-r} \\ & + \sum_{j=r+1}^{n-1} \frac{n!}{(j-1)!(n-j)!}f_X(x) \left[F_X(x)\right]^{j-1}\left[1 - F_X(x)\right]^{n-j} \\ &- \sum_{j=r}^{n-2} \frac{n!}{j![n-(j+1)]!}f_X(x) \left[F_X(x)\right]^j\left[1 - F_X(x)\right]^{n-j-1}\\ =&~\frac{n!}{(r-1)!(n-r)!}f_X(x) \left[F_X(x)\right]^{r-1}\left[1 - F_X(x)\right]^{n-r} \end{aligned} \end{equation}