I'm currently trying to understand more about the properties of the maximum likelihood estimator. It's known that, in the large data-limit, the MLE becomes an unbiased estimator with almost Gaussian distribution with minimum variance (it's efficient). In math, if we note $\theta_0$ the true value generating the data and $\hat \theta$ the MLE (and $I_0$ the Fisher information of the data-generating process):

$$ \sqrt n (\hat \theta - \theta_0) \rightarrow N(0, (n I_0)^{-1} ) $$

The conventional proof relies on computing the distribution of the empirical likelihood function, but are there other possibilities ? In particular, I'm wondering if somebody as come up with an explanation using information theory ideas.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.