# Prove that $\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2}$ is an unbiased estimate of $\text{Var}(\overline{X})$

If $$X_{1},X_{2},\ldots,X_{n}$$ are independent random variables with common mean $$\mu$$ and variances $$\sigma^{2}_{1},\sigma^{2}_{2},\ldots,\sigma^{2}_{n}$$. Prove that \begin{align*} \frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2} \end{align*} is an unbiased estimate of $$\text{Var}[\overline{X}]$$.

MY ATTEMPT

To begin with, I tried to approach the problem as it follows \begin{align*} \text{Var}(\overline{X}) = \text{Var}\left(\frac{X_{1} + X_{2} + \ldots + X_{n}}{n}\right) = \frac{\sigma^{2}_{1} + \sigma^{2}_{2} + \ldots + \sigma^{2}_{n}}{n^{2}} \end{align*}

But I do not know how to proceed from here. Any help is appreciated.

• @BruceET, in this case, $V(\bar{X}) \ne {\sigma^2\over{n}}$. There is no common variance. – StatsStudent Apr 20 at 22:19
• @StatsStudent. Right. Deleting comment. – BruceET Apr 20 at 22:24

## 3 Answers

Let's minimize the algebra. We can do this by focusing on the coefficient of $$\sigma_1^2.$$

First, because all the variables have the same mean,

$$E[X_i - \bar X]=\mu - \mu = 0$$

for all $$i,$$ implying

$$\operatorname{Var}(X_i - \bar X) = E[(X_i - \bar X)^2] - E[X_i - \bar X]^2 = E[(X_i - \bar X)^2].$$

Since the sum of the $$(X_i-\bar X)^2$$ is invariant under any permutation of the indexes, let's study the case $$i=1$$ because it will show us what happens with all the $$i.$$

We can easily split $$X_1 - \bar X$$ into independent (and therefore uncorrelated) parts as

$$X_1 - \bar X = X_1 - \frac{1}{n}\left(X_1 + X_2 + \cdots + X_n\right) = \frac{n-1}{n}\left(X_1 - \frac{1}{n-1}\left(X_2 + \cdots + X_n\right)\right).$$

Taking variances immediately gives

$$\operatorname{Var}(X_1 - \bar X) = \left(\frac{n-1}{n}\right)^2 \left(\sigma_1^2 + \frac{1}{(n-1)^2}\left(\sigma_2^2 + \cdots + \sigma_n^2\right)\right).$$

(This is the payoff from observing that the expectation of the square of $$X_i - \bar X$$ is its variance.)

When summing over all $$i$$ and ignoring the common factor of $$((n-1)/n)^2,$$ $$\sigma_1^2$$ will therefore appear once and it will appear $$n-1$$ more times with a factor of $$1/(n-1)^2,$$ for a total coefficient of

$$1 + (n-1)\left(\frac{1}{(n-1)^2}\right) = \frac{n}{n-1}.$$

Consequently every $$\sigma_i^2$$ appears with this coefficient, whence

\eqalign{ \frac{1}{n(n-1)}\sum_{i=1}^{n}(X_i - \bar{X})^{2} &= \frac{1}{n(n-1)} \sum_{i=1}^{n}\operatorname{Var}(X_i - \bar{X}) \\ &= \frac{1}{n(n-1)} \left(\frac{n-1}{n} \right)^2 \left[ \frac{n}{n-1} \sum_{i=1}^n \sigma_i^2 \right] \\ &= \frac{\sigma_1^2 + \sigma_2^2 + \cdots + \sigma_n^2}{n^2}, }

QED.

I would start from

\begin{align} \operatorname{E}(X_i-\overline X)^2&=\operatorname{E}[(X_i-\mu)-(\overline X-\mu)]^2 \\&=\operatorname{E}(X_i-\mu)^2+\operatorname{E}(\overline X-\mu)^2-2\operatorname{E}(X_i-\mu)(\overline X-\mu) \\&=\sigma_i^2+\operatorname{Var}(\overline X)-2\operatorname{Cov}(X_i,\overline X) \end{align}

So that,

\begin{align} \sum_{i=1}^n \operatorname{E}(X_i-\overline X)^2&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-2\sum_{i=1}^n \operatorname{Cov}(X_i,\overline X) \\&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-2\sum_{i=1}^n \operatorname{Cov}\left(X_i,\frac{1}{n}\sum_{j=1}^n X_j\right) \\&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-\frac{2}{n}\sum_{i= 1}^n\sum_{j=1}^n \operatorname{Cov}(X_i,X_j) \end{align}

Now I would try to write this expression in terms of $$\operatorname{Var}(\overline X)$$ only.

I'd recommend proceeding as follows to avoid having to work with covariances:

1. You have correctly calculated $$Var(\bar{X})$$.
2. Expand the term $$\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2}$$ and simplify.
3. Take the expectation of the result of step 2.
4. Reduce what you found in step 3 by using by substituting in for $$E(\cdot^2)$$ terms using what you know about the definition of a $$Var(\cdot)$$.
5. Reduce further and show that step 4 is equal to what you have already calculated for $$Var(\bar{X})$$.
6. Having shown the expected value of the term in step 2 is equal to $$Var(\bar{X})$$, you will have proven it's an unbiased estimator or $$Var(\bar{X})$$.

This will help you get started with at step 2:

$$\begin{eqnarray*} E\left[\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i}-\bar{X})^{2}\right] & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}\left(X_{i}^{2}-2\bar{X}X_{i}+\bar{X}^{2}\right)\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}\sum_{i=1}^{n}X_{i}+\sum_{i=1}^{n}\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}\frac{n}{n}\sum_{i=1}^{n}X_{i}+n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}^{2}n+n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}\left[\sum_{i=1}^{n}E\left(X_{i}^{2}\right)-nE\left(\bar{X}^{2}\right)\right] \end{eqnarray*}$$

From here, proceed to step 4 and the rest should follow pretty easily if you follow my instructions after some algebra.