7
$\begingroup$

Let $X_1,\dots,X_n$ be i.i.d. random variables with $X_1 \sim N(\mu,\sigma^2)$. Let $\bar X =\sum_{i=1}^n X_i/n$ and $R = X_{(n)}-X_{(1)}$, where $X_{(i)}$ is the $i$ the order statistic. Show that $\bar X$ and $R$ are independently distributed.

I know that sample mean and sample variance of normal distribution are independent. But this result state that sample mean and sample range of normal distribution are also independent. I know that $\bar X \sim N(\mu,\sigma^2/n)$.

$\endgroup$
  • $\begingroup$ for $n=2$, you can express the sample range $R$ as a function of the sample variance $S^2$, which proves it in that case. $\endgroup$ – kjetil b halvorsen May 6 '15 at 17:25
8
$\begingroup$

This is evidently a self-study question, so I do not intend to deprive you of the satisfaction of developing your own answer. Moreover, I'm sure there are many possible solutions. But for guidance, consider these observations:

  1. When a random variable $X$ is independent of other random variables $Y_1, \ldots, Y_m$, then $X$ is independent of any function of them, $f(Y_1, \ldots, Y_m)$. (See Functions of Independent Random Variables for more about this.)

  2. Because the $X_i$ are jointly Normal, $X_1 + \cdots + X_n$ is independent of all the differences $Y_{ij} = X_i - X_j$, since their covariances are zero.

Because the range can be expressed as

$$X_{(n)} - X_{(1)} = \max_{i,j}(|X_i - X_j|) = \max_{i,j}(|Y_{ij}|)$$

you can exploit (1) and (2) to finish the proof.


For more intuition, a quick simulation might be of some help. The following shows the marginal and joint distribution of the mean and range in the case $n=3$, using $10,000$ independent datasets. The joint distribution clearly is not bivariate Normal, so the temptation to prove independence by means of a zero correlation--although a good idea--is bound to fail. However, a close analysis of these results ought to suggest that the conditional distribution of the range does not vary with the mean. (The appearance of some variation at the right and left is due to the paucity of outcomes with such extreme means.)

Figure

Here is the R code that produced these figures. It is easily modified to vary $n$, vary the simulation size, and to analyze the simulation results more extensively.

n <- 3; n.sim <- 1e4
sim <- apply(matrix(rnorm(n * n.sim), n), 2, function(y) c(mean(y), diff(range(y))))
par(mfrow=c(1,3))
hist(sim[1,], xlab="Mean", main="Histogram of Means")
hist(sim[2,], xlab="Range", main="Histogram of Ranges")
plot(sim[1,], sim[2,], pch=16, col="#00000020", xlab="Mean", ylab="Range")
$\endgroup$
4
$\begingroup$

This exercise is an immediate application of Basu's theorem.

To begin with, first note that $\bar{X}$ and $R$ are independent if and only if $\sigma^{-1}\bar{X}$ and $\sigma^{-1}R$ are independent, so we may assume $X_1, \ldots, X_n \text{ i.i.d.} \sim N(\mu, 1)$.

It is well-known that $\bar{X}$ is a sufficient and complete statistic for $\mu$, so according to Basu's theorem, to show that $\bar{X}$ and $R$ are independent, it remains to show $R$ is ancillary, i.e., $R$'s distribution is independent of $\mu$. This is easily seen by noticing $$R = X_{(n)} - X_{(1)}= (X_{(n)} - \mu) - (X_{(1)} - \mu).$$ Clearly, the distribution of $(X_{(1)} - \mu, \ldots, X_{(n)} - \mu)$ is identical to the distribution of $(Z_{(1)}, \ldots, Z_{(n)})$, where $Z_1, \ldots, Z_n \text{ i.i.d.} \sim N(0, 1)$, which is distribution-constant. Consequently, the distribution of $R$ does not depend on $\mu$. This completes the proof.

$\endgroup$
2
$\begingroup$

A well-known property of the normal distribution is that the joint distribution of the differences $X_i-\overline X$, $i=1,2,\ldots,n-1$ is independent of the distribution of sample mean $\overline X$. This is also apparent from the independence of $\overline X=\frac{1}{n}\sum\limits_{i=1}^n X_i$ with the sample variance $\frac{1}{n-1}\sum\limits_{i=1}^n (X_i-\overline X)^2$, giving us $\operatorname{Cov}(X_i-\overline X,\overline X)=0$ for each $i$. The joint normality of $\overline X$ and $X_i-\overline X$ then implies their independence. Since the sample range $R=\max(X_i-\overline X)-\min(X_i-\overline X)$, $i=1,2,\ldots,n$ is a measurable function of $X_i-\overline X$ for each $i$, we can justify that $\overline X$ is independent of $R$.


An alternative proof of this result in line with @whuber's answer appears in this paper by J. Daly.

The argument provided in this article roughly goes along these lines:

We assume without loss of generality that $\mu=0$ and $\sigma^2=1$.

The joint characteristic function of $\overline X$ and the $\frac{n(n-1)}{2}$ differences $X_j-X_k$, $j<k$, is then

$$\varphi(t,t_{jk})=\frac{1}{(2\pi)^{n/2}}\int_{\mathbb{R^n}}\exp\left[-\frac{1}{2}\sum_{j=1}^nx_j^2+i\frac{t}{n}\sum_{j=1}^nx_j+i\sum_{1\le j<k\le n}t_{jk}(x_j-x_k)\right]\mathrm{d}\mathbf x$$

Completing the square in the exponent and further simplification leads to

$$\varphi(t,t_{jk})=\exp\left[-\frac{1}{2}\sum_{j=1}^n\left(\frac{t}{n}+\sum_{k=1}^n\left(t_{jk}-t_{kj}\right)\right)^2\right]$$

, which factors into the marginal characteristic functions

$$\varphi(t)\varphi(t_{jk})=\exp\left(-\frac{t^2}{2n}\right).\exp\left[-\frac{1}{2}\sum_{j=1}^n\left(\sum_{k=1}^n\left(t_{jk}-t_{kj}\right)\right)^2\right]$$

Thus the differences $X_j-X_k$ are jointly independent of $\overline X$.

Since the sample range $R=\max|X_j-X_k|$ is a measurable function of these differences, it follows that $\overline X$ and $R$ are independently distributed.

$\endgroup$
-5
$\begingroup$

Show that $\mathbf{X}=(X_1,\dots,X_n)' \sim N_n(\mathbf{\mu},\Sigma)$ where $\mathbf{\mu}=(\mu,\dots,\mu)'$ and $\Sigma = \begin{pmatrix} \sigma^2 & 0 & \dots & 0 \\0 & \sigma^2 & \dots & 0 \\ \vdots & \vdots & &\vdots & \\ 0 & 0 & \dots & \sigma^2\\ \end{pmatrix}$

Now note that $\bar X= (\frac{1}{n},\dots,\frac{1}{n})\mathbf{X}= \mathbf{a'}\mathbf{X}$. Also WLOG take $X_1=X_{(1)}$ and $X_n= X_{(n)}$. Then $R= (-1,0,\dots,0,1)\mathbf{X}= \mathbf{b'}\mathbf{X}$

Then note that $$\mathbf{a'}\Sigma \mathbf {b} = -\dfrac{\sigma^2}{n}+\dfrac{\sigma^2}{n} = 0 $$

This implies sample mean and sample range of normal distribution are independent.

$\endgroup$
  • 2
    $\begingroup$ It appears you are invoking an untrue theorem: the lack of correlation between two random variables does not imply independence unless they are jointly normal--but these are not. The range does not have a Normal distribution. $\endgroup$ – whuber May 6 '15 at 17:27
  • 2
    $\begingroup$ The range is not a linear transformation, @AdamO, because neither $X_{(n)}$ nor $X_{(1)}$ are linear functions of the $X_i$. (Isn't it clear that $\max$ and $\min$ are nonlinear functions?) $\endgroup$ – whuber May 6 '15 at 17:32
  • 2
    $\begingroup$ I have reluctantly downvoted this answer because it is incorrect and you have been unable to recognize or acknowledge that, which only risks confusing the OP and other readers. $\endgroup$ – whuber May 6 '15 at 17:39
  • 1
    $\begingroup$ @whuber I think I was trying (unsuccessfully) to allude to the WLOG argument being wrong. It is not right to assume that $X_1 = X_{(1)}$. It is an interesting question. I will mull about it a bit. $\endgroup$ – AdamO May 6 '15 at 17:49
  • 1
    $\begingroup$ The distribution of $X_{(n)}$ is given by the following argument: ${\rm Prob}[ X_{(n)} \le x] = {\rm Prob} [\mbox{ all of } X_1, \ldots, X_n \le x] = \bigl \{ {\rm Prob}[ X_1 \le x] \bigr\}^n $. From that, you can derive its cdf. Reverse the argument for the distribution of $X_{(1)}$. That is to show, at the very least, that $X_{(n)}$ is not normal. $\endgroup$ – StasK May 6 '15 at 19:41

protected by kjetil b halvorsen Apr 17 '18 at 13:19

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.