4
$\begingroup$

How to compute the density function of the k-th order statistic of a sample of $X_1, X_2, ..., X_n$ random variables distributed independently but not identically (i.e., $X_i \sim F_i$ with $F_i\neq F_j$)? What would be the explicit solution if each $F_i$ is uniform on $[m_i,1]$ with $0<m_i<m_{i+1}<...<m_n<1$?

$\endgroup$
  • 2
    $\begingroup$ As illustrated in a related thread, there is no nice simple expression in general: the situation can get quite complicated. $\endgroup$ – whuber Jun 5 '14 at 14:36
  • $\begingroup$ Do you know the distribution of each of the $X_i$? $\endgroup$ – wolfies Jun 5 '14 at 18:15
  • $\begingroup$ Yes. I do know the distribution $F_i$ for each $X_i$. If not, I can at least order them by a first-order stochastic dominance relation (e.g., $F_1 > F_2 > ...$). $\endgroup$ – mrb Jun 5 '14 at 18:47
  • $\begingroup$ To be honest, if I had a practical problem of this sort, unless the known distributions were very simple in form (and possibly even then), I'd probably be using simulation. $\endgroup$ – Glen_b Jun 6 '14 at 1:49
  • $\begingroup$ @mrb wrote: Yes. I do know the distribution Fi for each Xi ... So nu? What are they? If you provide them, you may get an answer to your specific problem. $\endgroup$ – wolfies Jun 7 '14 at 16:55
6
$\begingroup$

Preamble

As per the above comments, order statistics from non-identical distributions typically require complicated calculations, and generally yield complicated solutions, which makes them well-suited for solving with computer algebra systems. I am not aware that one can generally derive closed-form solutions as a function of the sample size $n$ and $k$-th order statistic (though it may be possible for your example)... but one CAN certainly obtain quite neat solutions to your problem, given any arbitrary integer value for $n$ and $k$ of your own choice. I am going to pursue the computer algebra approach here, because those tools are familiar to me, and because it makes short shrift of a lot of messy algebra.

The Problem

Let $X_i$ denote a continuous random variable with pdf $f(x; m_i)$, such that $(X_1,X_2,\dots,X_n)$ are independent but not identical variables due to differing parameters $m_i$, for $i = 1,\dots,n$. For the OP's question, we have a $Uniform(m,1)$ parent where identicality is relaxed by replacing parameter $m$ with $m_i$, for $i = 1, \dots, n$. Thus, the pdf $f(x; m_i)$, can be written:

enter image description here

To illustrate, here is plot of the family of pdf's, when $n = 4$, and $m_i = \frac{i}{5}$.

enter image description here

Solution

If $X_i$ has pdf $f(x; m_i)$, then, for any sample size $n$, the pdf of the $k$-th order statistic is given by:

$\qquad \qquad $OrderStatNonIdentical[k, {$f_i$}, {n}]

where OrderStatNonIdentical is a function from the mathStatica package for Mathematica, and where $n$ and $k$ are integers. For the OP's question, in a sample of size $n = 4$, the pdf of the 2nd smallest order statistic is given immediately by:

enter image description here

Here is a plot of the pdf of the 2nd order statistic (just derived), when the sample size is $n=4$, and $m_i = i/5$:

enter image description here

  • Similarly, the pdf of the 3rd order statistic is:

enter image description here

... and here is a plot of same:

enter image description here

Monte Carlo check

When doing symbolic work, it is always a good idea to check one's work using numerical methods, to make sure no errors have crept in. Here is a quick Monte Carlo check of the $k = 2$ case solution derived above, again with $m_i = i/5$, and $n = 4$. The ragged blue line is the empirical pdf (blue), plotted on top of the theoretical solution (dashed red line) derived above:

enter image description here

All looks fine :)

General Solutions by Induction

It appears possible to attain general symbolic solutions by induction, at least for the 1st and 2nd order statistics. In particular:

  • the pdf of the 1st order statistic, irrespective of the size of $n$, has form:

$$ \begin{cases} \frac{n(1-x)^{n-1}}{\prod_{i=1}^n (1-m_i)} & m_n < x < 1 \\ \quad \quad \dots & \quad \quad \dots \\ \frac{3(1-x)^2}{(1-m_1)(1-m_2)(1-m_3)} & m_3 < x \leq m_4 \\ \frac{2(1-x)}{(1-m_1)(1-m_2)} & m_2 < x \leq m_3 \\ \frac{1}{1-m_1} & m_1 < x \leq m_2 \\ 0 & \text{otherwise} \end{cases}$$

  • the pdf of the 2nd order statistic has form:

$$ \begin{cases} \frac{(n-1)(1-x)^{n-2}}{\prod_{i=1}^n (1-m_i)}(n x-\sum_{i=1}^n m_i) & m_n < x < 1 \\ \quad \quad \dots & \quad \quad \dots \\ \frac{2(1-x)^1}{(1-m_1)(1-m_2)(1-m_3)} (3x-m_1-m_2-m_3) & m_3 < x \leq m_4 \\ \frac{1(1-x)^0}{(1-m_1)(1-m_2)} (2x-m_1-m_2) & m_2 < x \leq m_3 \\ 0 & \text{otherwise} \end{cases}$$

$\endgroup$
  • $\begingroup$ +1 That's a computational tour de force. To my eye, though, there does not seem to be any pattern or closed form emerging as $n$ increases. What aspects of these solutions suggest otherwise to you? $\endgroup$ – whuber Jun 9 '14 at 19:25
  • 1
    $\begingroup$ Hi @whuber. The $k=2$ case looked pretty, but methinks it was lucky in its prettiness. Do agree that finding a pattern may be rather ahem challenging :) $\endgroup$ – wolfies Jun 9 '14 at 19:32
  • 1
    $\begingroup$ @whuber - Had another look at this ... one can get apparently general solutions for the 1st and 2nd order stats, for arbitrary $n$. This is rather fortunate, because whereas calculating the $(k= 2, n = 4)$ case takes just 1.7 seconds to evaluate, the $(k= 2, n = 6)$ case already takes 750 seconds! $\endgroup$ – wolfies Jun 10 '14 at 18:55
  • $\begingroup$ @mrb This is a nice answer. If it helped you you can accept it by clicking on the check mark underneath the voting buttons. Upvoting and accepting answers is an important signal for future readers with the same problem. $\endgroup$ – Andy Dec 9 '14 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.