3
$\begingroup$

Studying the asymptotic distribution of order statistics I came across this approximation:

$$F \left( E \left[ Y_n^{\left(n \right)} \right] \right) \approx E \left[ F \left( (Y_n^{\left( n \right)} \right) \right] $$

where $Y_n^{\left( n \right) }$ denotes the sequence of the nth order statistics and $F(.)$ is the distribution function of the underlying distribution.

I understand that because of the Probability Integral Transform, $F \left( Y_n^{ \left(n \right)} \right)$ is the largest observation from a sample of size $n$ from a uniform distribution over $(0,1)$.

Intuitively it makes sense to claim that the area to the left of the mean of $Y_n^{ \left( n \right) } $ is approximately equal to the mean of the area to the left of $Y_n^{ \left( n \right) } $. Is that really the case though? Also does this approximation hold for other order stastistics as well?

All help is greatly appreciated, thank you.

$\endgroup$
3
  • 1
    $\begingroup$ It's a good question, but in the interest of pursuing what might be an important issue, I would like to point out that the much simpler approximation $F(\mathbb{E}(Y_n^{(n)}))\approx 1/2$ already works pretty well: it is never off by more than $1/2$! The point, of course, is that it would behoove us to quantify the accuracy of your approximation and to establish a threshold to distinguish good from poor accuracy for your application. Do you have any thoughts about that? (BTW, the right hand side has a Beta distribution for any order statistic provided $F$ is a continuous distribution.) $\endgroup$
    – whuber
    Jun 27 '14 at 17:15
  • $\begingroup$ @whuber Indeed, it can be routinely derived that for the nth order statistic the RHS equals $\frac{n}{n+1}$. The authors use this approximation and the fact that the RHS can be computed to get an approximation for $E \left[ Y_n \right] $ which we need as our centering constant for the asymptotic distribution of $Y_n$ as sometimes it can hard to compute explicitly. $\endgroup$
    – JohnK
    Jun 27 '14 at 17:24
  • 1
    $\begingroup$ The asymptotic distribution of $Y_n^{(n)}$ will be an extreme value distribution. I don't think it's important (theoretically) to have an accurate estimate of its center, but I suppose in practice with finite $n$ such an estimate might help, depending on the application. $\endgroup$
    – whuber
    Jun 27 '14 at 17:40
3
$\begingroup$

It's the first term in a Taylor expansion - when taking only one term, it's a zero-order approximation.

It's quite common with Taylor expansions for moments of functions of random variables to take a second-order approximation (which would be 3 terms of the expansion, but when taking expectations the second term is zero).

The suggested Taylor approximation will only be accurate in some particular circumstances. I don't think that it will typically be the case for largest order statistics, but it might happen at least sometimes. The next non-zero term in the Taylor expansion should give some sense of the size of the bias.

$\endgroup$
4
  • $\begingroup$ Very useful technique, thank you for pointing this out. $\endgroup$
    – JohnK
    Jun 27 '14 at 17:32
  • $\begingroup$ By the way many of the examples I have been studying that use this approximation have a standard logistic distribution as the underlying pdf. Thus the approximation does not seem too bad, considering that $\sigma^2=\pi^2/3$ $\endgroup$
    – JohnK
    Jun 27 '14 at 17:45
  • 1
    $\begingroup$ +1 It might be worth pointing out that the error of the $n$th Taylor expansion can be understood in terms of the $(n+1)$th derivative. See here. This gives a case-by-case basis for determining how useful the approximation is in a worst case scenario. $\endgroup$
    – user44764
    Jun 27 '14 at 18:04
  • 1
    $\begingroup$ @Matthew +1 However, in this particular case (where we take expectations of an expansion about the mean), the (0+1)th derivative is 0 (so the zero order approximation is accurate to first order), -- and our investigation of the error passes to the next term. $\endgroup$
    – Glen_b
    Jun 27 '14 at 23:55
3
$\begingroup$

Jensen's Inequality states that for a random variable $Y$ and a convex function $\varphi$, we have $$ \varphi(E[Y]) \leq E[\varphi(Y)]. $$ If $\varphi$ is concave, then the direction of the inequality is switched. Furthermore, equality holds for non-constant $Y$ if and only if $\varphi$ is linear.

For large $n$, we expect $Y_n^{(n)}$ to be in the right tail of the distribution, where the distribution function $F$ is approximately linear for many common distributions. I believe this is where the given approximation would come from.

$\endgroup$
1
  • $\begingroup$ Indeed, thank you. It would seem that the approximation does quite well in the right tail of $F$; it might not be unreasonable to expect it to hold for $Y_1$ as well as many cdf's are also linear in the left tail. $\endgroup$
    – JohnK
    Jun 27 '14 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.