# can the maximum likelihood estimator depend on the parametrization?

That is, say I have a distribution with parameter $$\theta$$.

If I re-write it with a parameter $$a$$ such that $$a^3=\theta$$, is it possible that doing maximum likelihood estimation on a will yield an estimate $$\hat a$$ such that $$\hat a^3 \neq \hat \theta$$?

Could it be the case for another function different than $$x^3$$?

If so, what are some criteria to choose a parametrization?

• An estimate is a realization of the estimator.Usually, the realization of an estimator does not equal the target, $\hat\theta\neq\theta$. This is because a particular property of a sample usually does not equal the corresponding property of the population. Hence, it would not at all be surprising if $(\hat{a})^3\neq\theta$ even if $(\hat{a})^3$ is an estimator targeting $\theta$. Is your question perhaps whether it is possible that $(\hat{a}^{\text{MLE}})^3\neq\hat\theta^{\text{MLE}}$? – Richard Hardy Aug 17 '20 at 11:18
• – kjetil b halvorsen Aug 18 '20 at 1:41

## 1 Answer

The Invariance Property of Maximum Likelihood Estimators (MLEs) says, if $$\hat{\theta}$$ is the MLE of $$\theta$$, then for any function $$\tau(\theta)$$ the MLE of $$\tau(\theta)$$ is $$\tau(\hat{\theta})$$.

So, if you define $$a^3=\theta$$, once you obtained your MLE for $$\theta$$, $$\hat{\theta}$$, you can apply the inverse function by taking the cubed root of $$\hat{\theta}$$ and obtain the MLE of $$a$$ (i.e. $$\hat{a}=\hat{\theta}^{1\over{3}}$$)

Update:

I've added the proof mentioned by Thomas Lumley in the comments:

Let $$\hat{\eta}$$ denote the value that maximizes $$L^*(\eta|\textbf{x})$$. We must show that $$L^*(\hat{\eta}|\textbf{x})$$=$$L^*(\tau(\hat{\theta})|\textbf{x})$$. The maxima of $$L$$ and $$L^*$$ coincide, so we have

$$\begin{eqnarray*} L^{*}(\hat{\eta}|\textbf{x}) & = & \underset{\eta}{\text{sup}}\underset{\{\theta:\tau(\theta)=\eta\}}{\text{sup}}\,L(\theta|\textbf{x})\\ & = & \underset{\theta}{\text{sup}}L(\theta|\textbf{x})\\ & = & L(\hat{\theta}|\textbf{x}), \end{eqnarray*}$$

The first and third equalities hold by definition of $$L^{*}$$ and $$\hat{\theta}$$ respectively, and the second equality holds because the iterated maximization is equal to the unconditional maximization over $$\theta$$, obtained at $$\hat{\theta}$$. Further,

$$\begin{eqnarray*} L(\hat{\theta}|\textbf{x}) & = & \underset{\{\theta:\tau(\theta)=\tau(\hat{\theta})\}}{\text{sup}}L(\theta|\textbf{x})\\ & = & L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]. \end{eqnarray*}$$

Hence, the string of equalities shows that $$L^{*}(\hat{\eta}|\textbf{x})=L^{*}\left[\tau(\hat{\theta})|\textbf{x}\right]$$ and that $$\tau(\hat{\theta})$$ is the MLE of $$\tau(\theta)$$. $$\blacksquare$$

• Perhaps worth noting that the equivariance property holds even if the transformation isn't invertible. – Thomas Lumley Aug 17 '20 at 0:40
• Good point, @ThomasLumley. I don't think I made that clear and, in fact, may have obfuscated that point with my example (or the OP's example). – StatsStudent Aug 17 '20 at 0:45
• Wow. Very cool (and a bit surprising!). No conditions on tau? Do you happen to have a good statement of the theorem – josinalvo Aug 17 '20 at 2:58
• The proof is much more straightforward for invertible transformations. This question math.stackexchange.com/questions/3246587/… gives a reference to the text by Casella and Berger for the general result. – Thomas Lumley Aug 17 '20 at 5:12
• @ThomasLumley, I've added proof under the Update heading in case the OP does not have access to the book. – StatsStudent Aug 17 '20 at 5:32