Are my empirical findings correct? How to get the same result analytically?

I studied the efficiency of the mean and standard dev estimators:

$$\mu_n=\sum \frac {x_i} {n}\space\space\space\space\sigma_n=\sqrt{\frac 1 {n-1}\sum (x_i-\mu_n)^2}\space\space\space\space\ x_i\sim N\left[\mu,\sigma^2\right]$$

I ran several numerical simulations, varying the parameters: $$\space\space\space\space \mu=1,2,3,4 \space\space\space\space \sigma=5,10,15,20\space\space\space\space\ n=10,20,...,10000 $$

and compared the root mean percentage error (RMPE) of the two estimators:

$$\text{RMPE}_\mu= \sqrt{E\left[\left(\frac {\mu_n-\mu}{\mu}\right)^2\right]}$$ $$ \text{RMPE}_\sigma= \sqrt{E\left[\left(\frac {\sigma_n-\sigma}{\sigma}\right)^2\right]}$$

I discovered that the standard dev estimator is much more efficient than the mean estimator:


the ratio between the two seems not to vary with $n$, it only depends on $\mu$ and $\sigma$. Precisely: $$\frac {\text{RMPE}_\mu}{\text{RMPE}_\sigma}=\frac \sigma \mu \sqrt{2} $$

Are my empirical findings correct? How to get the same result analytically?

Thank you!

  • 1
    $\begingroup$ You can derive the RMPE of $\hat\sigma$ analytically from known formulas for the mean and variance of the en.wikipedia.org/wiki/Chi_distribution since $\hat\sigma^2(n-1)/\sigma^2$ is chi-square distributed. $\endgroup$ – Jarle Tufto Jan 22 at 13:53
  • 1
    $\begingroup$ The RMPE for the mean doesn't make a whole lot of sense: consider what it is estimating when $\mu\approx 0.$ Your "discovery" will change dramatically as you vary $\sqrt{n}\mu/\sigma$ (which was never very small in your simulations). $\endgroup$ – whuber Jan 22 at 17:45
  • $\begingroup$ @whuber I totally get your point! What do you suggest as alternative to compare the efficiency of different estimators, independently from their own specific scale? $\endgroup$ – elemolotiv Jan 22 at 18:14
  • $\begingroup$ I believe one standard way is to compare it to the efficiency (as usually defined in terms of variance) of the most efficient possible estimator in the class of estimators being considered. Ultimately what is relevant depends on the loss function in your particular application: in effect, that will tell you how to measure and compare the costs of estimating different parameters. $\endgroup$ – whuber Jan 22 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.