When is the inverse of the Fisher Information exact? (MLE)

Question

I just have a quick question about MLE.

Sometimes, when doing MLE problems, I see that the variance expression gotten from the inverse of the Fisher Information is exactly like what it should be and sometimes it isn't. Is there a reason that for some distributions this method is exact? Also, why is it that for some distributions, for example like $geom0(\pi)$, is it difficult to find an exact formula for $var(\tilde\pi)$?

Thanks so much for your help!

EDIT:

$Geom0(\pi)$ is the geometric distribution where the PMF is $(1-\pi)^y\pi$ where $y=0,1,2,...\infty$ as opposed to $Geom1(\pi)$ where the PMF is $(1-\pi)^{(y-1)}\pi$ where $y=1,2,...\infty$. Sorry for the confusion!

Answer 1

Sometimes, when doing MLE problems, I see that the variance expression gotten from the inverse of the Fisher Information is exactly like what it should be and sometimes it isn't.

All depends on what 'exactly like it should be' means ;)

At any rate, I think you need to take a look at the Cramer-Rao bound again. The inverse of the fisher information only gives a lower bound on the variance of an unbiased estimator. An estimator which achieve this is called an efficient estimator, as it has the lowest possible variance while being unbiased.

But there's lots of unbiased estimators which do not achieve this bound. There's lots of estimators which are biased, but still useful in practise. For example, the maximum likelihood estimator of variance is biased, but still good enough a lot of the time. These are probably what you're encountering.

As for why it's sometimes difficult to find a closed form solution of estimates of variance, etc, I'll only say that it would be pretty astonishing if every probability distribution out there had nice, closed form expressions for the stuff we're interested in.