4
$\begingroup$

Suppose we have $X_1, \cdots, X_n \overset{\textrm{i.i.d.}}{\sim} \mathcal{N}(0, 1)$ with $n > 50$, and let $X_{(1)}, \cdots, X_{(n)}$ be the associated order statistics.

Are there any references pointing to formulas that specify (or estimate) the variance of the $k^{\textrm{th}}$ order statistic, i.e. $\textrm{Var}(X_{(k)})$, for $1 \leq k \leq n$?

I am looking for formulas can be explicitly evaluated without resorting back to numerical integration.


I have seen this question: Approximate order statistics for normal random variables, which attracted a number of answers on approximate formula for the mean of the $k^{\textrm{th}}$ order statistic, e.g. Blom (1958) (and modifications based on work by Harter (1961) and Elfving (1947)).

Searching on Google also yields a number of work on the variance / standard deviation for the $k^\textrm{th}$ order statistics for small $n$, all of them opting for a computation/tabulation route. They are: Godwin(1949), Teichroew (1956), and Parrish (1992).

I have also attempted to derive something myself following the method sketched by @probabilityislogic in this question, along the lines of:

$$\textrm{Var}(X_{(k)}) = \int_{-\infty}^{\infty} \left(x-\mathbb{E}(X)\right)^2 \frac{n!}{(k-1)!(n-k)!} f_X(x)[1-F_X(x)]^{n-k}[F_X(x)]^{k-1}\;\textrm{d}x$$, with lower case $f$ denoting the PDF of a r.v. $X$, and upper case $F$ denoting the CDF.

Noting $x = F^{-1}(F_X(x))$ and using the substitution $u = F_X(x)$ (with corresponding r.v. in upper case $U$), we have: $$\textrm{Var}(X_{(k)}) = \int_{0}^{1} \left(F_X^{-1}(u) - \mathbb{E}\left(F_X^{-1}(U)\right)\right)^2\,\mathcal{B}(u | k, n-k+1) \; \textrm{d}u$$, where $\mathcal{B}$ denotes the PDF of a beta distribution (not the beta function). This can be rewritten as the variance based on the beta distribution:

$$\textrm{Var}(X_{(k)}) = \textrm{Var}_{\mathcal{B}(u | k, n-k+1)}\left(F_X^{-1}(u)\right)$$

Approximating the RHS using (the variance bit of the) delta method, we get: $$\textrm{Var}_{\mathcal{B}(u | k, n-k+1)}\left(F_X^{-1}(u)\right) \approx \textrm{Var}_{\mathcal{B}(u | k, n-k+1)}(U) \cdot \left[(F_X^{-1})'\left(\mathbb{E}_{\mathcal{B}(u | k, n-k+1)}(U)\right)\right]^2$$, where the prime denotes the derivative.

The first term of the product is standard result. For the second part, given $(F^{-1})'(\cdot)=\frac{1}{f(F^{-1})(\cdot)}$ (see e.g. this), we then have: $$\textrm{Var}_{\mathcal{B}(u | k, n-k+1)}\left(F_X^{-1}(u)\right) \approx \frac{k(n-k+1)}{(n+1)^2(n+2)} \frac{1}{\left(f\left(F^{-1}\left(\frac{k}{n+1}\right)\right)\right)^2}$$

Substituting in the standard normal distribution ($\phi$ for PDF, $\Phi$ for CDF) and scaling it to the desired variance $\sigma^2$ we arrive at: $$\textrm{Var}(X_{(k)}) \approx \frac{k(n-k+1)}{(n+1)^2(n+2)} \frac{1}{\left(\phi\left(\Phi^{-1}\left(\frac{k}{n+1}\right)\right)\right)^2}$$, which we can throw in a $\sigma^2$ term in front if it is not a standard normal.

But surely someone must have derived that somewhere (again I am looking for some references), and there is a high chance that I have gone off track somewhere above.

$\endgroup$
1
$\begingroup$

After two weeks I found someone had indeed derived the approximation above. It is in page 120 of their book [1] and page 12 of their accompanying course material [2].

The author stated the variance of the $k^{\textrm{th}}$ order statistics can be estimated as:

$$\textrm{Var}(X_{(k)}) \approx \frac{p(1-p)}{(n+2)(f(\theta))^2},$$

where $f(\cdot)$ is the PDF of $X$, $p = \frac{k}{n+1}$, and $\theta$ is the $p^\textrm{th}$ quantile of the distribution.

Applying to the normal case, we have $\theta = \Phi^{-1} (\frac{k}{n+1})$, and one can easily verify the referenced variance estimate equates to the variance derived in the original question with some rearrangement of terms.

[1] Jenny A. Baglivo (2005) Mathematica laboratories for mathematical statistics: Emphasizing simulation and computer intensive methods.
[2] Jenny A. Baglivo (2018) MATH4427 Notebook 4 - Fall Semester 2017/2018 - Boston College. URL: https://www2.bc.edu/jenny-baglivo/MT427/notebook04.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.