9
$\begingroup$

Assume we have access to i.i.d. samples from a distribution with true (unknown) mean and variance $\mu, \sigma^2$, and we want to estimate $\mu^2$.

How can we construct an unbiased, always positive estimator of this quantity?

Taking the square of the sample mean $\tilde{\mu}^2$ is biased and will overestimate the quantity, esp. if $\mu$ is close to 0 and $\sigma^2$ is large.

This is possibly a trivial question but my google skills are letting me down as estimator of mean-squared only returns mean-squarred-error estimators


If it makes matters easier, the underlying distribution can be assumed to be Gaussian.


Solution:

  • It is possible to construct an unbiased estimate of $\mu^2$; see knrumsey's answer
  • It is not possible to construct an unbiased, always positive estimate of $\mu^2$ as these requirement are in conflict when the true mean is 0; see Winks' answer
$\endgroup$
  • $\begingroup$ Perhaps seach for estimator of squared mean or estimator of square of mean instead. When I read your title, I was also confused (just like Google), so I edited it to make it more intuitive. $\endgroup$ – Richard Hardy Apr 7 '18 at 8:17
9
$\begingroup$

Note that the sample mean $\bar{X}$ is also normally distributed, with mean $\mu$ and variance $\sigma^2/n$. This means that $$\operatorname E(\bar{X}^2) = \operatorname E(\bar{X})^2 + \operatorname{Var}(\bar{X}) = \mu^2 + \frac{\sigma^2}n$$

If all you care about is an unbiased estimate, you can use the fact that the sample variance is unbiased for $\sigma^2$. This implies that the estimator $$\widehat{\mu^2} = \bar{X}^2 - \frac{S^2}n$$ is unbiased for $\mu^2$.

$\endgroup$
  • 2
    $\begingroup$ Thanks for your input! This is a good observation but does not satisfy the always positive requirement; given the samples {-1,1}, the sample mean is 0 and the sample variance is 2, leading to an estimate $\hat{\mu^2}$ of -1. $\endgroup$ – Winks Apr 7 '18 at 5:37
  • 3
    $\begingroup$ Given that $(\bar{X},S^2)$ is minimal sufficient and complete, this unbiased estimator should be the one with minimal variance. $\endgroup$ – Xi'an Apr 8 '18 at 7:37
  • $\begingroup$ @Winks That is the very reason why this is an example of an absurd unbiased estimator. $\endgroup$ – StubbornAtom Jun 28 '18 at 20:41
13
$\begingroup$

It should not be possible produce an estimator that is both unbiased and always positive for $\mu^2$.

If the true mean is 0, the estimator must in expectation return 0 but is not allowed to output negative numbers, therefore it is also not allowed to output positive numbers either as it would be biased. An unbiased, always positive estimator of this quantity must therefore always return the correct answer when the mean is 0, regardless of the samples, which seems impossible.

knrumsey's answer shows how to correct the bias of the sample-mean-squarred estimator to obtain an unbiased estimate of $\mu^2$.

$\endgroup$
  • 2
    $\begingroup$ There is a rather old paper by Jim Berger establishing this fact, but I cannot trace it. The problem also appears in Monte Carlo with debiasing estimators like Russian Roulette. $\endgroup$ – Xi'an Apr 7 '18 at 7:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.