I have a question about a proof (about MLE) but let me firstly give you appropriate context.

An estimator T is called maximum likelihood estimator of $\theta$, if: \begin{equation}L(T(\boldsymbol{x});\boldsymbol{x}) = max\space{} L(\theta ;\boldsymbol{x}) \end{equation}


if $\gamma = g(\theta)$ and $g$ is bijective, i.e $\theta = g^{-1}(\gamma)$, then $\hat{\theta}$ is a MLE for $\theta$ iff $g(\hat{\theta}) = \hat{\gamma}$ is a MLE for $\gamma$

The proof: The likelihood function is $L(\theta;\boldsymbol{x}) = p(\boldsymbol{x};\theta)$. since $g$ is bijective the likelihood function with respect to $\gamma$ is given by $\tilde{L}(\gamma;\boldsymbol{x})=p(\boldsymbol{x};g^{-1}(\gamma))$. Furthermore,we have

$\tilde{L}( \hat{\gamma};\boldsymbol{x}) \geq \tilde{L}( \gamma;\boldsymbol{x})$ for all $\gamma$

$\Leftrightarrow p(\boldsymbol{x};g^{-1}(\hat{\gamma})) \geq p(\boldsymbol{x};g^{-1}(\gamma))$ for all $\gamma$

$\Leftrightarrow p(\boldsymbol{x}:\hat{\theta}) \geq p(\boldsymbol{x};\theta)$ for all $\theta$

$\Leftrightarrow L(\hat{\theta};\boldsymbol{x}) \geq L(\theta;\boldsymbol{x})$ for all $\theta$

The thing that is bothering me about the proof is this statement:

since $g$ is bijective the likelihood function with respect to $\gamma$ is given by $\tilde{L}(\gamma;\boldsymbol{x})=p(\boldsymbol{x};g^{-1}(\gamma))$

But in this case $\tilde{L}(\gamma ;\boldsymbol{x}) = p(\boldsymbol{x}:\theta )$ which is a function of $\theta$ but how can that be? I am missing something important...


Because $p$ is taken to be a function of $\theta$ and not $\gamma$. So if we want to write the likelihood in terms of $p$ and the parameter $\gamma$, then we need to plug in the value for $\theta$ corresponding to $\gamma$, which is $g^{-1}(\gamma)$.

For instance, take a single observation $x$ from an exponential$(\lambda)$ distribution, where $\lambda$ is the rate parameter. The likelihood function is then $L(\lambda) = \lambda e^{- \lambda x}$. But the likelihood expressed in terms of the mean parameter $\mu = 1 / \lambda$ (so $g(t) = 1 / t$) is $\tilde{L}(\mu) = e^{- x / \mu} / \mu$, which is simply the original likelihood evaluated at $g^{-1}(\mu) = 1 / \mu$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.