5
$\begingroup$

I just have a quick question about MLE.

Sometimes, when doing MLE problems, I see that the variance expression gotten from the inverse of the Fisher Information is exactly like what it should be and sometimes it isn't. Is there a reason that for some distributions this method is exact? Also, why is it that for some distributions, for example like $geom0(\pi)$, is it difficult to find an exact formula for $var(\tilde\pi)$?

Thanks so much for your help!

EDIT:

$Geom0(\pi)$ is the geometric distribution where the PMF is $(1-\pi)^y\pi$ where $y=0,1,2,...\infty$ as opposed to $Geom1(\pi)$ where the PMF is $(1-\pi)^{(y-1)}\pi$ where $y=1,2,...\infty$. Sorry for the confusion!

$\endgroup$
  • $\begingroup$ What does $geom0$ mean? $\endgroup$ – Glen_b -Reinstate Monica Feb 24 '14 at 4:56
  • $\begingroup$ I have updated the question and hope that clarifies the difference! $\endgroup$ – nicefella Feb 24 '14 at 17:06
  • $\begingroup$ My apologies. Things should be correct now! $\endgroup$ – nicefella Feb 25 '14 at 2:41
3
$\begingroup$

Sometimes, when doing MLE problems, I see that the variance expression gotten from the inverse of the Fisher Information is exactly like what it should be and sometimes it isn't.

All depends on what 'exactly like it should be' means ;)

At any rate, I think you need to take a look at the Cramer-Rao bound again. The inverse of the fisher information only gives a lower bound on the variance of an unbiased estimator. An estimator which achieve this is called an efficient estimator, as it has the lowest possible variance while being unbiased.

But there's lots of unbiased estimators which do not achieve this bound. There's lots of estimators which are biased, but still useful in practise. For example, the maximum likelihood estimator of variance is biased, but still good enough a lot of the time. These are probably what you're encountering.

As for why it's sometimes difficult to find a closed form solution of estimates of variance, etc, I'll only say that it would be pretty astonishing if every probability distribution out there had nice, closed form expressions for the stuff we're interested in.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.