1
$\begingroup$

Let's say that for some observation I build a likelihood function. Then the point of maximum of that function will give the ML estimate of the parameters.

Is there a way to quantify the "confidence" (I am not sure if this is the right term here) of the estimate from the likelihood function? Indeed, I would expect that a more "impulse-like" likelihood function will give an estimate with higher confidence (more robust), while an almost flat one will be less robust.

I could normalise the likelihood function such that its area is one, and then take the entropy. But that doesn't seem like a very elegant approach. Alternatively, I could use some type of measure of flatness of the likelihood function. Is there a more elegant way of doing this?

$\endgroup$
1
$\begingroup$

Maximum likelihood estimates have their own central limit theorem when regularity conditions are met^1. Roughly, the ML CLT states that $\sqrt{n}\left( \hat{\theta}_{ML} - \theta \right) \rightarrow_d \mathcal{N}\left(0, \mathcal{I}(\theta)^{-1}\right)$. Where $\mathcal{I}(\theta)$ is the Fisher information, the double derivative of the log likelihood function. Geometrically, this means that if the likelihood function is "steep" at either side of the maxima, the information is large, and its inverse small. The converse holds as well. So your intuition is correct.

An approximate 95% confidence interval for a maximum likelihood estimate can be obtained by $\hat{\theta}_{ML} \pm \mathcal{Z}_{\alpha/2} \sqrt{\mathcal{I}(\hat{\theta}_{ML})^{-1}}$. A wider interval is produced by a flatter likelihood as you expect.

^1 There are irregular likelihoods for which this isn't true.

$\endgroup$
1
  • $\begingroup$ Thanks @AdamO ! That's exactly what I need. Continuing on intuition, though: what if my likelihood function has well separated peaks at almost exactly the same height (this is indeed my case). The Fisher information is still a local measure, so it won't see these alternative options. How do we reconcile this? I suppose this has to do with the fact that due to CLT the likelihood function becomes mono-modal. But with the amount of data I have I don't get to that point. Any other ideas, then? $\endgroup$
    – Enzo
    Mar 31 '18 at 10:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.