4
$\begingroup$

That is, say I have a distribution with parameter $\theta$.

If I re-write it with a parameter $a$ such that $a^3=\theta$, is it possible that doing maximum likelihood estimation on a will yield an estimate $\hat a$ such that $\hat a^3 \neq \hat \theta$?

Could it be the case for another function different than $x^3$?

If so, what are some criteria to choose a parametrization?

$\endgroup$
2

1 Answer 1

8
$\begingroup$

The Invariance Property of Maximum Likelihood Estimators (MLEs) says, if $\hat{\theta}$ is the MLE of $\theta$, then for any function $\tau(\theta)$ the MLE of $\tau(\theta)$ is $\tau(\hat{\theta})$.

So, if you define $a^3=\theta$, once you obtained your MLE for $\theta$, $\hat{\theta}$, you can apply the inverse function by taking the cubed root of $\hat{\theta}$ and obtain the MLE of $a$ (i.e. $\hat{a}=\hat{\theta}^{1\over{3}}$)

Update:

I've added the proof mentioned by Thomas Lumley in the comments:

Let $\hat{\eta}$ denote the value that maximizes $L^*(\eta|\textbf{x})$. We must show that $L^*(\hat{\eta}|\textbf{x})$=$L^*(\tau(\hat{\theta})|\textbf{x})$. The maxima of $L$ and $L^*$ coincide, so we have

\begin{eqnarray*} L^{*}(\hat{\eta}|\textbf{x}) & = & \underset{\eta}{\text{sup}}\underset{\{\theta:\tau(\theta)=\eta\}}{\text{sup}}\,L(\theta|\textbf{x})\\ & = & \underset{\theta}{\text{sup}}L(\theta|\textbf{x})\\ & = & L(\hat{\theta}|\textbf{x}), \end{eqnarray*}

The first and third equalities hold by definition of $L^{*}$ and $\hat{\theta}$ respectively, and the second equality holds because the iterated maximization is equal to the unconditional maximization over $\theta$, obtained at $\hat{\theta}$. Further,

\begin{eqnarray*} L(\hat{\theta}|\textbf{x}) & = & \underset{\{\theta:\tau(\theta)=\tau(\hat{\theta})\}}{\text{sup}}L(\theta|\textbf{x})\\ & = & L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]. \end{eqnarray*}

Hence, the string of equalities shows that $L^{*}(\hat{\eta}|\textbf{x})=L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]$ and that $\tau(\hat{\theta})$ is the MLE of $\tau(\theta)$. $\blacksquare$

$\endgroup$
6
  • 2
    $\begingroup$ Perhaps worth noting that the equivariance property holds even if the transformation isn't invertible. $\endgroup$ Commented Aug 17, 2020 at 0:40
  • $\begingroup$ Good point, @ThomasLumley. I don't think I made that clear and, in fact, may have obfuscated that point with my example (or the OP's example). $\endgroup$ Commented Aug 17, 2020 at 0:45
  • $\begingroup$ Wow. Very cool (and a bit surprising!). No conditions on tau? Do you happen to have a good statement of the theorem $\endgroup$
    – josinalvo
    Commented Aug 17, 2020 at 2:58
  • 3
    $\begingroup$ The proof is much more straightforward for invertible transformations. This question math.stackexchange.com/questions/3246587/… gives a reference to the text by Casella and Berger for the general result. $\endgroup$ Commented Aug 17, 2020 at 5:12
  • 2
    $\begingroup$ @ThomasLumley, I've added proof under the Update heading in case the OP does not have access to the book. $\endgroup$ Commented Aug 17, 2020 at 5:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.