# Distribution of 'unmixed' parts based on order of the mix

Suppose I have paired observations drawn i.i.d. as $X_i \sim \mathcal{N}\left(0,\sigma_x^2\right), Y_i \sim \mathcal{N}\left(0,\sigma_y^2\right),$ for $i=1,2,\ldots,n$. Let $Z_i = X_i + Y_i,$ and denote by $Z_{i_j}$ the $j$th largest observed value of $Z$. What is the (conditional) distribution of $X_{i_j}$? (or equivalently, that of $Y_{i_j}$)

That is, what is the distribution of $X_i$ conditional on $Z_i$ being the $j$th largest of $n$ observed values of $Z$?

I am guessing that as $\rho = \frac{\sigma_x}{\sigma_y} \to 0$, the distribution of $X_{i_j}$ converges to just the unconditional distribution of $X$, while as $\rho \to \infty$, the distribution of $X_{i_j}$ converges to the unconditional distribution of the $j$th order statistic of $X$. In the middle, though, I am uncertain.

• I removed the "mixture" tag because this is a question about a sum (or, equivalently, about correlated Normal variables), not about a mixture of them. – whuber May 5 '11 at 21:23
• $X_i$ is also assumed independent of $Y_i$, yes? – cardinal May 8 '11 at 0:14
• @cardinal: yes, they are independent. – shabbychef May 8 '11 at 2:33
• A recent and related question that popped up on math.SE: math.stackexchange.com/questions/38873/… – cardinal May 13 '11 at 12:55
• The solution posted on math.SE is conceptually identical to the solution I give below - but formulated using a slightly different terminology. – NRH May 14 '11 at 7:28

Observe that the random variable $i_j$ is a function of $\mathbf{Z} = (Z_1, \ldots, Z_n)$ only. For an $n$-vector, $\mathbf{z}$, we write $i_j(\mathbf{z})$ for the index of the $j$th largest coordinate. Let also $P_z(A) = P(X_1 \in A \mid Z_1 = z)$ denote the conditional distribution of $X_1$ given $Z_1$.

If we break probabilities down according to the value of $i_j$ and desintegrate w.r.t. $\mathbf{Z}$ we get

$$\begin{array}{rcl} P(X_{i_j} \in A) & = & \sum_{k} P(X_k \in A, i_j = k) \\ & = &\sum_k \int_{(i_j(z) = k)} P(X_k \in A \mid \mathbf{Z} = \mathbf{z}) P(\mathbf{Z} \in d\mathbf{z}) \\ & = & \sum_k \int_{(i_j(z) = k)} P(X_k \in A \mid Z_k = z_k) P(\mathbf{Z} \in d\mathbf{z}) \\ & = & \sum_k \int_{(i_j(z) = k)} P_{z_k}(A) P(\mathbf{Z} \in d\mathbf{z}) \\ & = & \int P_{z}(A) P(Z_{i_j} \in dz) \\ \end{array}$$

This argument is quite general and relies only on the stated i.i.d. assumptions, and $Z_k$ could be any given function of $(X_k, Y_k)$.

Under the assumptions of normal distributions (taking $\sigma_y = 1$) and $Z_k$ being the sum, the conditional distribution of $X_1$ given $Z_1 = z$ is $$N\left(\frac{\sigma_x^2}{1+\sigma_x^2} z, \sigma_x^2\left(1 - \frac{\sigma_x^2}{1+\sigma_x^2}\right)\right)$$ and @probabilityislogic shows how to compute the distribution of $Z_{i_j}$, hence we have explicit expressions for both the distributions that enter in the last integral above. Whether the integral can be computed analytically is another question. You might be able to, but off the top of my head I can't tell if it is possible. For asymptotic analysis when $\sigma_x \to 0$ or $\sigma_x \to \infty$ it might not be necessary.

The intuition behind the computation above is that this is a conditional independence argument. Given $Z_{k} = z$ the variables $X_{k}$ and $i_j$ are independent.

The distribution of $Z_{i_{j}}$ is not difficult, and it is given by the Beta-F compound distribution:

$$p_{Z_{i_{j}}}(z)dz=\frac{n!}{(j-1)!(n-j)!} \frac{1}{\sigma_{z}}\phi(\frac{z}{\sigma_{z}})\left[\Phi(\frac{z}{\sigma_{z}})\right]^{j-1}\left[1-\Phi(\frac{z}{\sigma_{z}})\right]^{n-j}dz$$

Where $\phi(x)$ is a standard normal PDF, and $\Phi(x)$ is a standard normal CDF, and $\sigma_{z}^{2}=\sigma_{y}^{2}+\sigma_{x}^{2}$.

Now if you are given that $Y_{i_{j}}=y$, then $X_{i_{j}}$ is a 1-to-1 function of $Z_{i_{j}}$, namely $X_{i_{j}}=Z_{i_{j}}-y$. So I would think that this should be a simple application of the jacobian rule.

$$p_{X_{i_{j}}|Y_{i_{j}}}(x|y)=\frac{n!}{(j-1)!(n-j)!} \frac{1}{\sigma_{z}}\phi(\frac{x+y}{\sigma_{z}})\left[\Phi(\frac{x+y}{\sigma_{z}})\right]^{j-1}\left[1-\Phi(\frac{x+y}{\sigma_{z}})\right]^{n-j}dx$$

This seems too easy, but I think it is correct. Happy to be shown wrong.

• You have misunderstood the question. I am looking for the distribution of $X_{i_j}$ as a function of $j, n, \sigma_x, \sigma_y$. I do not actually observe the $X_i$ and $Y_i$, and cannot condition on them. One may assume, w.l.o.g. that $\sigma_x = 1$, and thus consider only the parameters $j, n, \sigma_y$. – shabbychef May 9 '11 at 16:24
• ok - so basically you need to have $y$ removed from this equation? (integrated out) – probabilityislogic May 9 '11 at 22:11
• yes; and it is not independent of Z... – shabbychef May 9 '11 at 22:21