0
$\begingroup$

Question: Prove that $\hat{\sigma}_x^2=\displaystyle\frac{1}{N-1}\sum_{i=1}^N(X_i-\overline{X})^2$, with $\overline{X}=\frac{1}{N}\sum_{i=1}^N X_i$ is an unbiased, minimum variance estimator of the variance, with the variance of the estimator being $\operatorname{Var}[\hat{\sigma}_x^2]=\displaystyle\frac{N}{(N-1)^2}E[(X_i-\mu)^4]+\frac{\sigma_x^4}{N-1}$.


So far I have been able to show that $E[\sigma_x^2]=\sigma^2$, but for proving that it's indeed the estimator with the minimum variance, I'm stuck. Any help would be much appreciated.

$\endgroup$
11
  • 1
    $\begingroup$ Hello Subhasis. Welcome here. Have you checked through the relevant posts here? As common as this question is, there should be some thread(s) that could be helpful to you. Have you checked any? $\endgroup$ Commented Apr 21 at 10:24
  • $\begingroup$ @User1865345 I tried going through the relevant threads, but the answers do not seem to be exactly what I want (or they're usually beyond my current level of understanding). That's why I decided to make a fresh question. $\endgroup$ Commented Apr 21 at 10:28
  • 1
    $\begingroup$ Okay. Thanks for your update. $\endgroup$ Commented Apr 21 at 10:29
  • 1
    $\begingroup$ Also, you can add how the other threads couldn't help Or what aspect you didn't understand. Otherwise it would likely be closed as a duplicate. $\endgroup$ Commented Apr 21 at 10:36
  • 1
    $\begingroup$ Was the distribution mentioned or not? $\endgroup$ Commented Apr 21 at 11:41

1 Answer 1

2
$\begingroup$

I'm pretty sure the question was actually asking about the Normal case, but the general case is interesting (if unhelpful).

The statement is true, under various much weaker conditions. In order to make it even meaningful, I will assume that $X$ are iid and have finite variance. I will write $S^2$ for the estimator under consideration

To start with, we have

  1. $S^2$ is unbiased for the variance under any (finite-variance) distribution for $X$: you have shown this. So it is non-parametrically an unbiased estimator

  2. $S^2$ is MVUE for Normal $X$: this is shown in other threads. So no estimator other than $S^2$ can be MVUE non-parametrically: it must have worse variance than $S^2$ at the Normal.

We now have two possibilities: there is no MVUE or $S^2$ is the MVUE.

The reason $S^2$ is the MVUE is that there essentially no other unbiased estimators. This needs some qualification. There are silly estimators, such as $T_3$, which is $S^2$ applied to just the first three observations, or $T_{\text {pair}}$ which is obtained by applying $S$ to pairs of observations 1,2, then to observations 3,4, then to 5,6, ... up to $(n-1),n$ and averaging the $n/2$ resulting estimates. What there aren't is any genuinely different estimators, because these sets of distribution are too big to have any genuinely different unbiased estimator $T$ satisfy $E[T]=E[S^2]$ for all distributions.

That's the motivation. Proving it takes a bit more effort.

  1. Shuster in The American Statistician looks at the case where $X$ takes on only finitely many possible values $\{\xi_1,\dots,\xi_m\}$, so $X_i$ can be seen as samples with replacement from a finite population -- a multinomial distribution. Write $R_i$ for the number of copies of $\xi_i$ in the sample. The set $\{R_i\}$ is a complete sufficient statistic, and $S^2$ is a function of it, so no other unbiased estimator $T$ can also be a function of it. By the Lehmann-Scheffe theorem, $S^2$ is MVUE

  2. Along similar lines, if we assume only that the distribution of $X$ is absolutely continuous, then the order statistics are a complete sufficient statistic, and $S^2$ is a function of them, so $S^2$ is MVUE over this class of distributions

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.