Conditional expectation of uniform random variable given order statistics

Question

Assume X = $(X_1, ..., X_n)$ ~ $U(\theta, 2\theta)$, where $\theta \in \Bbb{R}^+$.

How does one calculate the conditional expectation of $E[X_1|X_{(1)},X_{(n)}]$, where $X_{(1)}$ and $X_{(n)}$ are the smallest and largest order statistics respectively?

My first thought would be that since the order statistics limit the range, it is simply $(X_{(1)}+X_{(n)})/2$, but im not sure if this is correct!

Answer 1

Consider the case of an iid sample $X_1, X_2, \ldots, X_n$ from a Uniform$(0,1)$ distribution. Scaling these variables by $\theta$ and translating them by $\theta$ endows them with a Uniform$(\theta, 2\theta)$ distribution. Everything relevant to this problem changes in the same way: the order statistics and the conditional expectations. Thus, the answer obtained in this special case will hold generally.

Let $1\lt k\lt n.$ By emulating the reasoning at https://stats.stackexchange.com/a/225990/919 (or elsewhere), find that the joint distribution of $(X_{(1)}, X_{(k)}, X_{(n)})$ has density function

$$f_{k;n}(x,y,z) = \mathcal{I}(0\le x\le y\le z \le 1) (y-x)^{k-2}(z-y)^{n-k-1}.$$

Fixing $(x,z)$ and viewing this as a function of $y,$ this is recognizable as a Beta$(k-1, n-k)$ distribution that has been scaled and translated into the interval $[x,z].$ Thus, the scale factor must be $z-x$ and the translation takes $0$ to $x.$

Since the expectation of a Beta$(k-1,n-k)$ distribution is $(k-1)/(n-1),$ we find that the conditional expectation of $X_{(k)}$ must be the scaled, translated expectation; namely,

$$\mathbb{E}\left(X_{(k)}\mid X_{(1)}, X_{(n)}\right) = X_{(1)} + \left(X_{(n)}-X_{(1)}\right) \frac{k-1}{n-1}.$$

The cases $k=1$ and $k=n$ are trivial: their conditional expectations are, respectively, $X_{(1)}$ and $X_{(k)}.$

Let's find the expectation of the sum of all order statistics:

$$\mathbb{E}\left(\sum_{k=1}^n X_{(k)}\right) = X_{(1)} + \sum_{k=2}^{n-1} \left(X_{(1)} + \left(X_{(n)}-X_{(1)}\right) \frac{k-1}{n-1}\right) + X_{(n)}.$$

The algebra comes down to obtaining the sum $$\sum_{k=2}^{n-1}(k-1) = (n-1)(n-2)/2.$$

Thus

$$\eqalign{ \mathbb{E}\left(\sum_{k=1}^n X_{(k)}\right) &= (n-1)X_{(1)} + \left(X_{(n)}-X_{(1)}\right) \frac{(n-1)(n-2)}{2(n-1)} + X_{(n)} \\ &= \frac{n}{2}\left(X_{(n)}+X_{(1)}\right). }$$

Finally, because the $X_i$ are identically distributed, they all have the same expectation, whence

$$\eqalign{n\mathbb{E}\left(X_1\mid X_{(1)}, X_{(n)}\right) &= \mathbb{E}\left(X_1\right) + \mathbb{E}\left(X_2\right) + \cdots + \mathbb{E}\left(X_n\right)\\ &= \mathbb{E}\left(X_{(1)}\right) + \mathbb{E}\left(X_{(2)}\right) + \cdots + \mathbb{E}\left(X_{(n)}\right) \\ &= \frac{n}{2}\left(X_{(n)}+X_{(1)}\right), }$$

with the unique solution

$$\mathbb{E}\left(X_1\mid X_{(1)}, X_{(n)}\right) = \left(X_{(n)}+X_{(1)}\right)/2.$$

---

It seems worth remarking that this result is not a sole consequence of the symmetry of the uniform distribution: it is particular to the uniform family of distributions. For some intuition, consider data drawn from a Beta$(a,a)$ distribution with $a \lt 1.$ This distribution's probabilities are concentrated near $0$ and $1$ (its density has a U or "bathtub" shape). When $X_{(n)}\lt 1/2,$ we can be sure most of the data are piled up close to $X_{(1)}$ and therefore will tend to have expectations less than the midpoint $(X_{(1)}+X_{(n)})/2;$ and when $X_{(1)}\gt 1/2,$ the opposite happens and most of the data are likely piled up close to $X_{(n)}.$

Answer 2

The following is not a proof but a verification of the desired result once you know that $(X_{(1)},X_{(n)})$ is a complete statistic for $\theta$ :

Joint pdf of $X_1,X_2,\ldots,X_n$ is

\begin{align} f_{\theta}(x_1,\ldots,x_n)&=\frac{1}{\theta^n}\mathbf1_{\theta<x_{(1)},x_{(n)}<2\theta} \\&=\frac{1}{\theta^n}\mathbf1_{\frac{1}{2}x_{(n)}<\theta<x_{(1)}}\quad,\,\theta\in\mathbb R^+ \end{align}

So $T=(X_{(1)},X_{(n)})$ is a sufficient statistic for $\theta$. It can be shown that $T$ is also a complete statistic by proceeding along these lines.

Then by Lehmann-Scheffe theorem, $E\,[X_1\mid T]$ is the UMVUE of $E(X_1)=\frac{3\theta}{2}$.

Now, $\frac{1}{\theta}(X_i-\theta)\stackrel{\text{i.i.d}}\sim \mathsf U(0,1)$, so that $\frac{1}{\theta}(X_{(n)}-\theta)\sim\mathsf{Beta}(n,1)$ and $\frac{1}{\theta}(X_{(1)}-\theta)\sim \mathsf{Beta}(1,n)$.

Therefore, $E(X_{(n)})=\frac{n\theta}{n+1}+\theta=\frac{(2n+1)\theta}{n+1}$ and $E(X_{(1)})=\frac{\theta}{n+1}+\theta=\frac{(n+2)\theta}{n+1}$.

Hence,

$$E\left[\frac{1}{2}(X_{(1)}+X_{(n)})\right]=\frac{1}{2(n+1)}\left((n+2)\theta+(2n+1)\theta\right)=\frac{3\theta}{2}$$

This proves that $\frac{1}{2}(X_{(1)}+X_{(n)})$ is the UMVUE of $\frac{3\theta}{2}$ by Lehmann-Scheffe.

Since UMVUE is unique whenever it exists, it verifies the claim that $E\,[X_1\mid T]=\frac{1}{2}(X_{(1)}+X_{(n)})$.