That is, say I have a distribution with parameter $\theta$.

If I re-write it with a parameter $a$ such that $a^3=\theta$, is it possible that doing maximum likelihood estimation on a will yield an estimate $\hat a$ such that $\hat a^3 \neq \hat \theta$?

Could it be the case for another function different than $x^3$?

If so, what are some criteria to choose a parametrization?


1 Answer 1


The Invariance Property of Maximum Likelihood Estimators (MLEs) says, if $\hat{\theta}$ is the MLE of $\theta$, then for any function $\tau(\theta)$ the MLE of $\tau(\theta)$ is $\tau(\hat{\theta})$.

So, if you define $a^3=\theta$, once you obtained your MLE for $\theta$, $\hat{\theta}$, you can apply the inverse function by taking the cubed root of $\hat{\theta}$ and obtain the MLE of $a$ (i.e. $\hat{a}=\hat{\theta}^{1\over{3}}$)


I've added the proof mentioned by Thomas Lumley in the comments:

Let $\hat{\eta}$ denote the value that maximizes $L^*(\eta|\textbf{x})$. We must show that $L^*(\hat{\eta}|\textbf{x})$=$L^*(\tau(\hat{\theta})|\textbf{x})$. The maxima of $L$ and $L^*$ coincide, so we have

\begin{eqnarray*} L^{*}(\hat{\eta}|\textbf{x}) & = & \underset{\eta}{\text{sup}}\underset{\{\theta:\tau(\theta)=\eta\}}{\text{sup}}\,L(\theta|\textbf{x})\\ & = & \underset{\theta}{\text{sup}}L(\theta|\textbf{x})\\ & = & L(\hat{\theta}|\textbf{x}), \end{eqnarray*}

The first and third equalities hold by definition of $L^{*}$ and $\hat{\theta}$ respectively, and the second equality holds because the iterated maximization is equal to the unconditional maximization over $\theta$, obtained at $\hat{\theta}$. Further,

\begin{eqnarray*} L(\hat{\theta}|\textbf{x}) & = & \underset{\{\theta:\tau(\theta)=\tau(\hat{\theta})\}}{\text{sup}}L(\theta|\textbf{x})\\ & = & L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]. \end{eqnarray*}

Hence, the string of equalities shows that $L^{*}(\hat{\eta}|\textbf{x})=L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]$ and that $\tau(\hat{\theta})$ is the MLE of $\tau(\theta)$. $\blacksquare$

  • 2
    $\begingroup$ Perhaps worth noting that the equivariance property holds even if the transformation isn't invertible. $\endgroup$ Aug 17, 2020 at 0:40
  • $\begingroup$ Good point, @ThomasLumley. I don't think I made that clear and, in fact, may have obfuscated that point with my example (or the OP's example). $\endgroup$ Aug 17, 2020 at 0:45
  • $\begingroup$ Wow. Very cool (and a bit surprising!). No conditions on tau? Do you happen to have a good statement of the theorem $\endgroup$
    – josinalvo
    Aug 17, 2020 at 2:58
  • 3
    $\begingroup$ The proof is much more straightforward for invertible transformations. This question math.stackexchange.com/questions/3246587/… gives a reference to the text by Casella and Berger for the general result. $\endgroup$ Aug 17, 2020 at 5:12
  • 2
    $\begingroup$ @ThomasLumley, I've added proof under the Update heading in case the OP does not have access to the book. $\endgroup$ Aug 17, 2020 at 5:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.