7
$\begingroup$

Suppose we have the following random variables $X_1$, $X_2$,....$X_n$,.., that are $iid$ but we dont know what distribution they follow.

I know that the sample mean $\bar{X}$ is an unbiased estimator of the population mean. But, how can i prove that the square of the sample mean is an biased (or maybe unbiased) estimator of the variance?

My particular doubt is how to continue this:

$E[\bar{X}^2] = E[(\frac{\sum_{i=1}^nX_i}{n})^2] = E[\frac{\sum_{i=1}^nX_i}{n} \times\frac{\sum_{i=1}^nX_i}{n}] = \frac{1}{n^2} E[\sum_{i=1}^nX_i \times \sum_{i=1}^nX_i] = .....$

I think the estimator is biased, but i want to confirm it...

$\endgroup$
0

3 Answers 3

10
$\begingroup$

You have $X_1, X_2, \dots, X_n$ are iid from an unknown distribution with mean (say) $\mu$ and variance (say) $\sigma^2$.

$\bar{X}$ is an unbiased estimator of the mean, and thus $E(\bar{X}) = \mu$. Also, $Var(\bar{X}) = \sigma^2/n$. Thus since, \begin{align*} E[\bar{X}^2] & = Var(\bar{X}) + E[\bar{X}]^2\\ & = \dfrac{\sigma^2}{n} + \mu^2. \end{align*}

You can now figure out what the bias is. Clearly, $\bar{X}^2$ is a horrible estimator for $\sigma^2$. As wolfies pointed you, you will do better with $n\bar{X}^2$.

$\endgroup$
5
  • 1
    $\begingroup$ if I continue the OP's derivation, I get $n/n^2 E(X_i^2)=n(\mu^2+\sigma^2)/n^2=(\mu^2+\sigma^2)/n$. Where's my - presumably stupid - mistake? $\endgroup$ Commented Jun 8, 2021 at 15:16
  • 1
    $\begingroup$ @ChristophHanck Did you make the mistake of setting $E[(\sum X_i)^2 ] = n \sum E(X_i^2)$? $\endgroup$ Commented Jun 8, 2021 at 17:30
  • 1
    $\begingroup$ AFAICS, no. I multiply out the brackets, and, in expectation, only the terms with identical index are nonzero due to iidness, so $\frac{1}{n^2} E[\sum_{i=1}^nX_i \times \sum_{i=1}^nX_i]=\frac{1}{n^2} E[\sum_{i=1}^nX_i^2 ]$. I rearrange this to $\frac{1}{n^2} \sum_{i=1}^nE[X_i^2 ]=\frac{1}{n^2} \sum_{i=1}^n\sigma^2=\frac{n}{n^2} \sigma^2$ $\endgroup$ Commented Jun 9, 2021 at 4:04
  • 1
    $\begingroup$ @ChristophHanck These are not mean zero necessarily. So the terms with different indices $E(X_i X_j) = E(X_i) E(X_j) = \mu^2$. So they don't disappear. $\endgroup$ Commented Jun 9, 2021 at 5:30
  • 2
    $\begingroup$ argh...there we go. thanks. $\endgroup$ Commented Jun 9, 2021 at 6:16
3
$\begingroup$

Here is a solution using the 'Moment of Moment' functions in the mathStatica package for Mathematica. In particular, let $s_1$ denote the sample sum, i.e. $s_1 = \sum_{i=1}^nX_i$. Then, you seek

$$E\big[{\big(\frac{s_1}{n}\big)}^2\big]$$

which is the $1^{\text{st}}$ Raw Moment of $(\frac{s_1}{n})^2$, expressed here in terms of Central moments:

enter image description here

where $\mu_2$ denotes the $2^{\text{nd}}$ central moment of the population (i.e. the population variance). Plainly, this is a biased estimator of population variance.

Perhaps what you intended was $E\big[n {\big(\frac{s_1}{n}\big)}^2\big]$:

enter image description here

which will be an unbiased estimator of population variance, if the population mean is zero.

$\endgroup$
2
$\begingroup$

Do you actually mean something like "$\frac{1}{n-1} \sum_i \left(x_i - \bar{x} \right)^2$, where $\bar{x}$ is the sample mean, is an unbiased estimator of the population variance?" Or perhaps, "Is $\frac{1}{n} \sum_i x_i^2 - \bar{x}^2$ an unbiased estimator of the population variance?"

Trivial counterexample for what you literally asked:

It's trivial to show that the square of the sample mean is neither a consistent nor unbiased estimator in the general case.

Assume $X_i = 2$ for all i:

  • The sample mean is 2, no matter what.
  • The population variance is 0.
  • The sample mean squared is 4.
  • $ 4 \neq 0$

I'd bet though this isn't what the homework is asking for. (Assuming this is homework.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.