5
$\begingroup$

Let $X_1,..,X_n$ be an iid sample of $N(0,\sigma^2)$. Find an unbiased estimator of $\sigma^2$ and its lower bound.

I found that $$\hat{\sigma}^2 = \sum_{i=1}^{n} X_i^2$$ is an unbiased estimator for $\sigma^2$ and found that its Cramer-Rao Lower Bound is $\frac{1}{n^2}$. Additionally, I think that $\hat{\sigma}^2$ is a complete and sufficient statistics for $\sigma^2$ and hence an UMVUE.

Nonetheless, $$V(\sum_{i=1}^{n} X_i^2)=\sum_{i=1}^{n} V(X_i^2)=n V(X^2)=2n\sigma^4.$$ If the estimator is UMVUE, shouldn't its variance achieve the Cramer-Rao Lower Bound?

Improve this question
$\endgroup$
7
  • 3
    $\begingroup$ The sum of squares is not an unbiased estimator for $\sigma^2$. You need to divide by $n$. $\endgroup$
    – JohnK
    May 15, 2015 at 0:10
  • $\begingroup$ @JohnK I think you're thinking of consistency. Actually, you need to use Bessel's correction to get an unbiased estimator for $\sigma^2$ (see stats.stackexchange.com/questions/100041/…). $\endgroup$
    – mugen
    May 15, 2015 at 2:27
  • 1
    $\begingroup$ @mugen the mean here is known. Why would Bessel's correction be needed? $\endgroup$
    – Glen_b
    May 15, 2015 at 5:23
  • $\begingroup$ How did your calculation of the CR bound work? Why doesn't it change with changing $\sigma^2$? Can you show it? $\endgroup$
    – Glen_b
    May 15, 2015 at 5:24
  • $\begingroup$ @mugen Consistency? No no. Recall that $E(X^2)=\sigma^2+\mu^2$. Now take the expectation of the sum. $\endgroup$
    – JohnK
    May 15, 2015 at 8:49

1 Answer 1

Reset to default
5
$\begingroup$

Let's settle this. For a Normal distribution with zero mean and variance $\sigma^2$, the complete sufficient statistic is indeed given by the sum of squares.

By the Rao-Blackwell Theorem if we can find an unbiased function of this sufficient statistic, then we have an MVUE. Furthermore, since the family of this statistic is complete, by the Lehmann-Scheffe theorem, it is also the unique MVUE for $\sigma^2$. So let's find an unbiased function of $\displaystyle{\sum_{i=1}^n X_i ^2}$ for $X\sim N\left (0,\sigma^2 \right)$.

Recall that for any distribution for which these moments exist

$$E(X^2)=\sigma^2+\mu^2$$

which in turn suggests that for this random sample

$$E \left( \sum_{i=1}^n X_i^2 \right)=n\sigma^2$$

from which it immediately follows that the unbiased estimator in question is

$$\widehat{\sigma^2}=\frac{1}{n} \sum_{i=1}^n X_i^2$$

Now, let's find the variance of this estimator. We know that for $X\sim N\left (0,\sigma^2 \right)$,

$$\frac{\sum_{i=1}^n X_i^2}{\sigma^2}\sim \chi^2(n)$$

Be very mindful of the degrees of freedom of the $\chi^2$ distribution. What would happen if we didn't know the mean and used an estimate instead? By the properties of the $\chi^2$ distribution then,

$$var\left(\frac{\sum_{i=1}^n X_i^2}{\sigma^2} \right)=2n$$

and so $var\left( \sum_{i=1}^n X_i^2 \right)=2n\sigma^4$. Thus for our modified estimator $var \left( \frac{1}{n} \sum_{i=1}^n X_i^2\right)=\frac{2\sigma^4}{n}$ which equals the Cramer-Rao bound. This should be comforting, right?

As a final remark, I would like to point out that the Cramer-Rao bound is only attainable if the mean of the normal distribution is known, as in this situation. If that had not been the case, then we would have to settle for an estimator that does not achieve the lower bound of variance.

Hope this clears it up a bit.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, I had managed to get the answer, was calculating the fisher information with n observations $\endgroup$
    – user72621
    May 15, 2015 at 11:53
  • $\begingroup$ You mention that the variance equals the Cramer-Rao bound, but you never derive it. How does one arrive at the CR bound? $\endgroup$
    – JoeVictor
    Feb 10, 2019 at 18:25
  • $\begingroup$ I'm trying to understand what this means in practical terms. Am I understanding correctly that ${2 \sigma^4 \over n}$ is telling you how well you know the variance of your measurement? $\endgroup$
    – olliepower
    Apr 2, 2019 at 23:51
  • $\begingroup$ @olliepower $\frac{2\sigma^4}{n}$ is the minimal variance of the parameter $\sigma^2$ (regardless of which estimator you choose). This is the very best you can hope for with any estimator, as no estimator will have less variance than this. Luckily our estimator has the same minimal variance, and not more! So we are in good shape - as we have a really good estimator for that parameter. $\endgroup$
    – Maverick Meerkat
    Feb 13, 2020 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.