4
$\begingroup$

If $X_{1},X_{2},\ldots,X_{n}$ are independent random variables with common mean $\mu$ and variances $\sigma^{2}_{1},\sigma^{2}_{2},\ldots,\sigma^{2}_{n}$. Prove that \begin{align*} \frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2} \end{align*} is an unbiased estimate of $\text{Var}[\overline{X}]$.

MY ATTEMPT

To begin with, I tried to approach the problem as it follows \begin{align*} \text{Var}(\overline{X}) = \text{Var}\left(\frac{X_{1} + X_{2} + \ldots + X_{n}}{n}\right) = \frac{\sigma^{2}_{1} + \sigma^{2}_{2} + \ldots + \sigma^{2}_{n}}{n^{2}} \end{align*}

But I do not know how to proceed from here. Any help is appreciated.

$\endgroup$
0

3 Answers 3

3
$\begingroup$

Let's minimize the algebra. We can do this by focusing on the coefficient of $\sigma_1^2.$

First, because all the variables have the same mean,

$$E[X_i - \bar X]=\mu - \mu = 0$$

for all $i,$ implying

$$\operatorname{Var}(X_i - \bar X) = E[(X_i - \bar X)^2] - E[X_i - \bar X]^2 = E[(X_i - \bar X)^2].$$

Since the sum of the $(X_i-\bar X)^2$ is invariant under any permutation of the indexes, let's study the case $i=1$ because it will show us what happens with all the $i.$

We can easily split $X_1 - \bar X$ into independent (and therefore uncorrelated) parts as

$$X_1 - \bar X = X_1 - \frac{1}{n}\left(X_1 + X_2 + \cdots + X_n\right) = \frac{n-1}{n}\left(X_1 - \frac{1}{n-1}\left(X_2 + \cdots + X_n\right)\right).$$

Taking variances immediately gives

$$\operatorname{Var}(X_1 - \bar X) = \left(\frac{n-1}{n}\right)^2 \left(\sigma_1^2 + \frac{1}{(n-1)^2}\left(\sigma_2^2 + \cdots + \sigma_n^2\right)\right).$$

(This is the payoff from observing that the expectation of the square of $X_i - \bar X$ is its variance.)

When summing over all $i$ and ignoring the common factor of $((n-1)/n)^2,$ $\sigma_1^2$ will therefore appear once and it will appear $n-1$ more times with a factor of $1/(n-1)^2,$ for a total coefficient of

$$1 + (n-1)\left(\frac{1}{(n-1)^2}\right) = \frac{n}{n-1}.$$

Consequently every $\sigma_i^2$ appears with this coefficient, whence

$$\eqalign{ E \Bigg[ \frac{1}{n(n-1)}\sum_{i=1}^{n}(X_i - \bar{X})^{2} \Bigg] &= \frac{1}{n(n-1)} \sum_{i=1}^{n}\operatorname{Var}(X_i - \bar{X}) \\ &= \frac{1}{n(n-1)} \left(\frac{n-1}{n} \right)^2 \left[ \frac{n}{n-1} \sum_{i=1}^n \sigma_i^2 \right] \\ &= \frac{\sigma_1^2 + \sigma_2^2 + \cdots + \sigma_n^2}{n^2}, }$$

QED.

$\endgroup$
4
$\begingroup$

I would start from

\begin{align} \operatorname{E}(X_i-\overline X)^2&=\operatorname{E}[(X_i-\mu)-(\overline X-\mu)]^2 \\&=\operatorname{E}(X_i-\mu)^2+\operatorname{E}(\overline X-\mu)^2-2\operatorname{E}(X_i-\mu)(\overline X-\mu) \\&=\sigma_i^2+\operatorname{Var}(\overline X)-2\operatorname{Cov}(X_i,\overline X) \end{align}

So that,

\begin{align} \sum_{i=1}^n \operatorname{E}(X_i-\overline X)^2&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-2\sum_{i=1}^n \operatorname{Cov}(X_i,\overline X) \\&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-2\sum_{i=1}^n \operatorname{Cov}\left(X_i,\frac{1}{n}\sum_{j=1}^n X_j\right) \\&=\sum_{i=1}^n \sigma_i^2+n\operatorname{Var}(\overline X)-\frac{2}{n}\sum_{i= 1}^n\sum_{j=1}^n \operatorname{Cov}(X_i,X_j) \end{align}

Now I would try to write this expression in terms of $\operatorname{Var}(\overline X)$ only.

$\endgroup$
3
$\begingroup$

I'd recommend proceeding as follows to avoid having to work with covariances:

  1. You have correctly calculated $Var(\bar{X})$.
  2. Expand the term $\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i} - \overline{X})^{2}$ and simplify.
  3. Take the expectation of the result of step 2.
  4. Reduce what you found in step 3 by using by substituting in for $E(\cdot^2)$ terms using what you know about the definition of a $Var(\cdot)$.
  5. Reduce further and show that step 4 is equal to what you have already calculated for $Var(\bar{X})$.
  6. Having shown the expected value of the term in step 2 is equal to $Var(\bar{X})$, you will have proven it's an unbiased estimator or $Var(\bar{X})$.

This will help you get started with at step 2:

\begin{eqnarray*} E\left[\frac{1}{n(n-1)}\sum_{i=1}^{n}(X_{i}-\bar{X})^{2}\right] & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}\left(X_{i}^{2}-2\bar{X}X_{i}+\bar{X}^{2}\right)\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}\sum_{i=1}^{n}X_{i}+\sum_{i=1}^{n}\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}\frac{n}{n}\sum_{i=1}^{n}X_{i}+n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-2\bar{X}^{2}n+n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}E\left[\sum_{i=1}^{n}X_{i}^{2}-n\bar{X}^{2}\right]\\ & = & \frac{1}{n(n-1)}\left[\sum_{i=1}^{n}E\left(X_{i}^{2}\right)-nE\left(\bar{X}^{2}\right)\right] \end{eqnarray*}

From here, proceed to step 4 and the rest should follow pretty easily if you follow my instructions after some algebra.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.