2
$\begingroup$

In beginning stats, we were told that:

  • $\bar{x}$ is an unbiased estimate of $\mu$
  • $\frac{1}{n - 1}\sum(x - \bar{x})^2$ is an unbiased estimate of $\sigma^2$

As I am reading more, I have learned that you could estimate population variance and mean using other ways such as:

  • Method of Moments
  • Maximum Likelihood Estimation

To round out my knowledge, I have a few questions.

  1. Is there an exhaustive list of estimation techniques used to estimate population mean and variance? Or are those 2 the only ones?
  2. Is "$\bar{x}$ is an unbiased estimate of $\mu$" considered an example of method of moments estimation?
  3. What about "$\frac{1}{n - 1}\sum(x - \bar{x})^2$ is an unbiased estimate of $\sigma^2$"? What estimation technique does that involve? My understanding is both MLE and MOM produce biased estimates of population variance.
$\endgroup$
3
  • 1
    $\begingroup$ For 2 and 3, you can say $\mathbb E\left[\bar{x} \right]=\mu$ and $\mathbb E\left[\frac{1}{n-1}\sum (x_i-\bar{x})^2 \right]=\sigma^2$ and for normally distributed random variables with unknown mean and variance they are the "minimum variance unbiased estimators" (MVUE) related to the Rao–Blackwell theorem and Lehmann–Scheffé theorem. $\endgroup$
    – Henry
    May 22, 2020 at 12:15
  • $\begingroup$ On 1, all these and others, such as minimum mean square error, are frequentist point estimators. There are more. More broadly there are other techniques such as interval estimation and Bayesian methods with posterior distributions for the parameters (you can even have various Bayesian point estimators). There may not be an exhaustive list $\endgroup$
    – Henry
    May 22, 2020 at 12:20
  • $\begingroup$ The method of moments produces unbiased estimators of all non-central moments. $\endgroup$
    – Xi'an
    May 22, 2020 at 13:31

1 Answer 1

2
$\begingroup$

1) Exhaustive? Nope.

2) $\bar{x}$ is the MOM estimator and happens to be unbiased. Being unbiased and MOM do not go together, however.

3) The $n-1$ adjustment comes from taking the expected value of the MLE and finding that there’s an easy correction to get an unbiased estimator. When you take the square root to get what you might think is an unbiased estimator of standard deviation, you in fact get a biased estimator of standard deviation! There is not a straightforward adjustment for all distributions to give an unbiased estimator, though it has been worked out in special cases (such as normal).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.