1
$\begingroup$

When estimating the standard-deviation for samples of normally distributed data, it is sometimes necessary to account for bias in whatever estimator one chooses -- which is usually related to the number of observations in the sample.

For example if we take the corrected-sample standard deviation, $$s = \sqrt{\frac{1}{n-1} \sum_{i}^{N} \left(x_{i} - \bar{x} \right)^{2}}$$ we account for sample-size bias with $$C_{4}(N) = \sqrt{\frac{2}{N-1}}\frac{\Gamma(N/2)}{\Gamma((N-1)/2)}$$ with an unbiased estimator for the standard deviation being $$\sigma_{\rm{est}} = s / c_{4}(n).$$

For other estimators such as $\rm{MAD}$, $S_{n}$, and $Q_{n}$ there are also bias correction factors which also scale with the sample size.

Is there a similar correction factor one can apply when using maximum-likelihood methods? In the plot below I have simulated normally distributed, $ \mathcal{N}(0, 1)$, for different sample sizes and estimated the standard deviation with $s$, $s/c_{4}$, and with a maximum-likelihood evaluation. We can see the bias behaviour: enter image description here

$\endgroup$

1 Answer 1

2
+50
$\begingroup$

The difference between the MLE for the variance and $s^2$ is dividing by $N$ instead of by $N-1$. Therefore, the MLE is $\sqrt{\frac{N-1}{N}}s$. Hence, $$\sqrt{\frac{N}{N-1}}\times MLE/c_4(N)=s/c_4(N)$$ is unbiased.

$\endgroup$
2
  • 1
    $\begingroup$ Allow me to play around and I'll accept. $\endgroup$
    – N.B.
    Dec 31, 2020 at 21:35
  • 1
    $\begingroup$ Thanks for this, I'll award the bounty when the timer allows -- can you give me a source or name for this? I haven't been able to find this on Wikipedia. $\endgroup$
    – N.B.
    Jan 1, 2021 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.