Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
That is, say I have a distribution with parameter $\theta$.
If I re-write it with a parameter $a$ such that $a^3=\theta$, is it possible that doing maximum likelihood estimation on a will yield an estimate $\hat a$ such that $\hat a^3 \neq \hat \theta$?
Could it be the case for another function different than $x^3$?
If so, what are some criteria to choose a parametrization?
The Invariance Property of Maximum Likelihood Estimators (MLEs) says, if $\hat{\theta}$ is the MLE of $\theta$, then for any function $\tau(\theta)$ the MLE of $\tau(\theta)$ is $\tau(\hat{\theta})$.
So, if you define $a^3=\theta$, once you obtained your MLE for $\theta$, $\hat{\theta}$, you can apply the inverse function by taking the cubed root of $\hat{\theta}$ and obtain the MLE of $a$ (i.e. $\hat{a}=\hat{\theta}^{1\over{3}}$)
Update:
I've added the proof mentioned by Thomas Lumley in the comments:
Let $\hat{\eta}$ denote the value that maximizes $L^*(\eta|\textbf{x})$. We must show that $L^*(\hat{\eta}|\textbf{x})$=$L^*(\tau(\hat{\theta})|\textbf{x})$. The maxima of $L$ and $L^*$ coincide, so we have
\begin{eqnarray*} L^{*}(\hat{\eta}|\textbf{x}) & = & \underset{\eta}{\text{sup}}\underset{\{\theta:\tau(\theta)=\eta\}}{\text{sup}}\,L(\theta|\textbf{x})\\ & = & \underset{\theta}{\text{sup}}L(\theta|\textbf{x})\\ & = & L(\hat{\theta}|\textbf{x}), \end{eqnarray*}
The first and third equalities hold by definition of $L^{*}$ and $\hat{\theta}$ respectively, and the second equality holds because the iterated maximization is equal to the unconditional maximization over $\theta$, obtained at $\hat{\theta}$. Further,
\begin{eqnarray*} L(\hat{\theta}|\textbf{x}) & = & \underset{\{\theta:\tau(\theta)=\tau(\hat{\theta})\}}{\text{sup}}L(\theta|\textbf{x})\\ & = & L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]. \end{eqnarray*}
Hence, the string of equalities shows that $L^{*}(\hat{\eta}|\textbf{x})=L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]$ and that $\tau(\hat{\theta})$ is the MLE of $\tau(\theta)$. $\blacksquare$
