There are multiple versions of this question on CV and many proofs are given but I did not fully understand the proof (technique) yet.

Theorem. If $$\hat\theta = \operatorname*{argsup}_{\theta\in\Theta} L(\theta, x)$$ exists, then for any function $g:\Theta\rightarrow\Lambda$ it holds that $\hat\lambda:=g(\hat\theta)$ is the MLE of $\lambda:=g(\theta)$.

Proof. Put $\tilde L(\lambda,x):=\sup\limits_{\theta\in\Theta:g(\theta)=\lambda}L(\theta,x)$.

Then $$\begin{align*} \hat\lambda &= \operatorname*{argsup}_{\lambda\in\Lambda} \tilde L(\lambda,x)\\ &= \operatorname*{argsup}_{\lambda\in\Lambda}\sup_{\theta\in\Theta:g(\theta)=\lambda}L(\theta,x)\\ &= \ ?\\ &= g\left(\operatorname*{argsup}_{\theta\in\Theta}L(\theta,x)\right)\\ &= g(\hat\theta). \end{align*}$$


  • Why is $g$ not required to be measurable?
  • Why is $\hat\lambda = \operatorname*{argsup}_{\lambda\in\Lambda}\tilde L(\lambda,x)$?
  • What happens at "?" step?
  • (is this proof correct?)
  • 2
    $\begingroup$ The mathematical notation might be obscuring the fundamental simplicity of this result: namely, $g$ merely is a way of renaming the optimal value(s) of the parameter. $\endgroup$ – whuber Jun 18 '18 at 19:32
  • 1
    $\begingroup$ I think this is exactely what confuses me... The notion is clear to me and if I think about it makes sense, especially if one considers one-to-one $g$. However, I struggle to write it down in a formal correct way. This may not be the best analogy but when I was a first year calculus student, I also thought that $\sup f(x) = f(\sup x)$ makes totally sense but is not true. This exercise reminds me of this example, especially since it feels like interchanging function and supremum at some point... $\endgroup$ – Syd Amerikaner Jun 18 '18 at 20:58
  • $\begingroup$ Interesting example. The problem with equating $\sup f$ with $f(\sup x),$ of course, is that $\sup x$ might not even be in the domain of $f$! $\endgroup$ – whuber Jun 18 '18 at 21:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.