3
$\begingroup$

If $X_{1},X_{2},\ldots,X_{n}$ are independent random variables with common mean $\mu$ and variances $\sigma^{2}_{1},\sigma^{2}_{2},\ldots,\sigma^{2}_{n}$. Prove that \begin{align*} \frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2} \end{align*} is an unbiased estimate of $\text{Var}[\overline{X}]$.

MY ATTEMPT

To begin with, I tried to approach the problem as it follows \begin{align*} \text{Var}(\overline{X}) = \text{Var}\left(\frac{X_{1} + X_{2} + \ldots + X_{n}}{n}\right) = \frac{\sigma^{2}_{1} + \sigma^{2}_{2} + \ldots + \sigma^{2}_{n}}{n^{2}} \end{align*}

But I do not know how to proceed from here. Any help is appreciated.

$\endgroup$
  • 1
    $\begingroup$ @BruceET, in this case, $V(\bar{X}) \ne {\sigma^2\over{n}}$. There is no common variance. $\endgroup$ – StatsStudent Apr 20 at 22:19
  • 1
    $\begingroup$ @StatsStudent. Right. Deleting comment. $\endgroup$ – BruceET Apr 20 at 22:24
2
$\begingroup$

Let's minimize the algebra. We can do this by focusing on the coefficient of $\sigma_1^2.$

First, because all the variables have the same mean,

$$E[X_i - \bar X]=\mu - \mu = 0$$

for all $i,$ implying

$$\operatorname{Var}(X_i - \bar X) = E[(X_i - \bar X)^2] - E[X_i - \bar X]^2 = E[(X_i - \bar X)^2].$$

Since the sum of the $(X_i-\bar X)^2$ is invariant under any permutation of the indexes, let's study the case $i=1$ because it will show us what happens with all the $i.$

We can easily split $X_1 - \bar X$ into independent (and therefore uncorrelated) parts as

$$X_1 - \bar X = X_1 - \frac{1}{n}\left(X_1 + X_2 + \cdots + X_n\right) = \frac{n-1}{n}\left(X_1 - \frac{1}{n-1}\left(X_2 + \cdots + X_n\right)\right).$$

Taking variances immediately gives

$$\operatorname{Var}(X_1 - \bar X) = \left(\frac{n-1}{n}\right)^2 \left(\sigma_1^2 + \frac{1}{(n-1)^2}\left(\sigma_2^2 + \cdots + \sigma_n^2\right)\right).$$

(This is the payoff from observing that the expectation of the square of $X_i - \bar X$ is its variance.)

When summing over all $i$ and ignoring the common factor of $((n-1)/n)^2,$ $\sigma_1^2$ will therefore appear once and it will appear $n-1$ more times with a factor of $1/(n-1)^2,$ for a total coefficient of

$$1 + (n-1)\left(\frac{1}{(n-1)^2}\right) = \frac{n}{n-1}.$$

Consequently every $\sigma_i^2$ appears with this coefficient, whence

$$\eqalign{ \frac{1}{n(n-1)}\sum_{i=1}^{n}(X_i - \bar{X})^{2} &= \frac{1}{n(n-1)} \sum_{i=1}^{n}\operatorname{Var}(X_i - \bar{X}) \\ &= \frac{1}{n(n-1)} \left(\frac{n-1}{n} \right)^2 \left[ \frac{n}{n-1} \sum_{i=1}^n \sigma_i^2 \right] \\ &= \frac{\sigma_1^2 + \sigma_2^2 + \cdots + \sigma_n^2}{n^2}, }$$

QED.

$\endgroup$
3
$\begingroup$

I would start from

\begin{align} \operatorname{E}(X_i-\overline X)^2&=\operatorname{E}[(X_i-\mu)-(\overline X-\mu)]^2 \\&=\operatorname{E}(X_i-\mu)^2+\operatorname{E}(\overline X-\mu)^2-2\operatorname{E}(X_i-\mu)(\overline X-\mu) \\&=\sigma_i^2+\operatorname{Var}(\overline X)-2\operatorname{Cov}(X_i,\overline X) \end{align}

So that,

\begin{align} \sum_{i=1}^n \operatorname{E}(X_i-\overline X)^2&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-2\sum_{i=1}^n \operatorname{Cov}(X_i,\overline X) \\&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-2\sum_{i=1}^n \operatorname{Cov}\left(X_i,\frac{1}{n}\sum_{j=1}^n X_j\right) \\&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-\frac{2}{n}\sum_{i= 1}^n\sum_{j=1}^n \operatorname{Cov}(X_i,X_j) \end{align}

Now I would try to write this expression in terms of $\operatorname{Var}(\overline X)$ only.

$\endgroup$
3
$\begingroup$

I'd recommend proceeding as follows to avoid having to work with covariances:

  1. You have correctly calculated $Var(\bar{X})$.
  2. Expand the term $\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2}$ and simplify.
  3. Take the expectation of the result of step 2.
  4. Reduce what you found in step 3 by using by substituting in for $E(\cdot^2)$ terms using what you know about the definition of a $Var(\cdot)$.
  5. Reduce further and show that step 4 is equal to what you have already calculated for $Var(\bar{X})$.
  6. Having shown the expected value of the term in step 2 is equal to $Var(\bar{X})$, you will have proven it's an unbiased estimator or $Var(\bar{X})$.

This will help you get started with at step 2:

\begin{eqnarray*} E\left[\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i}-\bar{X})^{2}\right] & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}\left(X_{i}^{2}-2\bar{X}X_{i}+\bar{X}^{2}\right)\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}\sum_{i=1}^{n}X_{i}+\sum_{i=1}^{n}\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}\frac{n}{n}\sum_{i=1}^{n}X_{i}+n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}^{2}n+n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}\left[\sum_{i=1}^{n}E\left(X_{i}^{2}\right)-nE\left(\bar{X}^{2}\right)\right] \end{eqnarray*}

From here, proceed to step 4 and the rest should follow pretty easily if you follow my instructions after some algebra.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.