I understand that maximum likelihood estimation allows us to estimate the parameter of a distribution that maximises the probability of observations occurring, and that this is in essence a way of estimating what the true distribution of our data is.

I'm reading about the invariance property of maximum likelihood estimators i.e. if $\hat\theta_{ML}$ is a maximum likelihood estimator of $\theta$, then $g(\hat\theta_{ML}$) is the maximum likelihood estimator of $g(\theta)$, for some function $g$.

The notes I'm reading through then say "since $f_{\theta}$ depends on $\theta$, any interesting quantity (expected values, probabilities) will be a function of $\theta$. Therefore if we can find the MLE of $\theta$, then we can find the MLE of any of these quantities.

My question is what would the MLEs of these quantities represent exactly and why would they be of use to us?

E.g. say the MLE, $\hat\mu_{ML}$, of the expectation of the distribution $f_{\theta}$. I understand that $\hat\theta$ is the parameter value that best captures the data, but once we have this, why would we be interested in the MLE of the expectation?

  • 2
    $\begingroup$ Your language suggests you think of a parameter estimate as if it completely characterized the distribution--but except in the case of one-parameter families, that's not true. $\endgroup$ – whuber Feb 12 at 16:58
  • $\begingroup$ The EM algorithm comes to mind. $\endgroup$ – Scott Feb 12 at 16:59
  • 2
    $\begingroup$ Also log likelihood is a useful consequence of this fact. Sometimes finding the MLE of $\theta$ is hard, but finding the MLE of $\log \theta$ is easy. Fortunately, $\log \theta_{ML}$ is the maximum likelihood estimator of $\log \theta$, so we have an easy way of recovering $\theta_{ML}$ $\endgroup$ – Scott Feb 12 at 17:43

The MLE of any quantity from $f_{\theta}$ represent your inference of its true value, but it is not more informative than what you already learned from the inferred distribution $f_{\hat{\theta}_{ML}}$.

Once you have obtained $\hat{\theta}_{ML}$, you have an estimation $f_{\hat{\theta}_{ML}}$ of the true distribution $f_{\theta}$, from which you can compute quantities of interest : mean, variance, moments, etc.

For instance, let's imagine you want to fit a binomial distribution of parameters $N$ and $p$ on your data. You obtain estimation of the parameters $\hat{N}$ and $\hat{p}$, and the ML estimate of the mean is $\hat{N} \hat{p}$ (which is also the mean of your inferred distribution).

In short, once you have MLEs of the parameters, you have estimated your distributions AND any related quantities, but not JUST these quantities.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.