I am working on a problem where we are interested in finding the MLE for a function of two parameters.

I am having problems with going about finding this. Intuitively, the idea makes sense. I am just wondering about the definition of the MLE of a function of two parameters (Google isn't turning up much). The question is as follows:

Question: Suppose that $X_1,\ldots,\,X_n$ are iid $N(\mu, \sigma^2)$ with unknown $\mu,\sigma^2$. Find the MLE for $\frac{\mu}{\sigma}$.

Note that this is not just a homework problem, but part of a take home final. I really am not looking for much of an answer, but more or less the idea for such problems.


Apparently MLE's are invariant under function. TY

  • 3
    $\begingroup$ Assuming you have found the MLE of $\mu$ and $\theta$ you just need to use this theorem. $\endgroup$ – caburke Dec 10 '12 at 0:36

The question was answered with a link in a comment, so let me just give here the argument from the link, for future completeness.

We assume a statistical model for data $X$ is parameterized by a parameter $\theta$ (which can be scalar or vector, or even more general). Let the likelihood function be $L(\theta)$ and the value of $\theta$ maximizing that be the maximum likelihood estimator $\hat{\theta}$ (mle). We do assume that estimator exists and is unique. Wanted is the mle of $g(\theta)$, a function of $\theta$. First we assume that $g$ is one-to-one. Then we can write $$ L(\theta) = L(g^{-1}(g(\theta)) $$ and both functions are clearly maximized by $\hat{\theta}$, so $$ \hat{\theta} = g^{-1}(\hat{g(\theta)}) $$ or $$ g(\hat{\theta}) = \hat{g(\theta)} $$ If $g$ is many-to-one, then $\hat{\theta}$ which maximizes $L(\theta)$ still corresponds to $g(\hat{\theta})$, so $g(\hat{\theta})$ still corresponds to the maximum of $L(\theta)$. (Argument paraphrased from the link in the comment above).

| cite | improve this answer | |

I just came up with a solution that I personally like more (for wide range of reasonable estimators):

Assume that $f$ is convex (likelihood function, the solution is unique) and $A$ is monotonous (function of parameter, $\exists A^{-1}$), then $f(A(x))$ is also convex.

What we want to prove is that

$$ \arg\max_{x\in X} f(A(x)) = A^{-1}(\arg\max_{z \in A(X)} f(z))$$

Now assume that there exists such a pair $(\tilde x, \tilde z)$ s.t. they are optimal for corresponding problems but $A(\tilde x) \neq \tilde z$.

From optimality: $f(\tilde z) > f(z') \quad \forall z'$ and $f(A(\tilde x)) > f(A(x')) \quad \forall x'$

including $z' = A(\tilde x)$ and $x' = A^{-1}(\tilde z)$, then $$ f(\tilde z) > f(A(\tilde x)) > f(A(A^{-1}(\tilde z)) = f(\tilde z)$$ - contradiction!

| cite | improve this answer | |
  • $\begingroup$ more specifically, if you want to get some $\hat g = g(\theta)$ consider a function $A = g^{-1}$ and $f$ is the likelihood $f(\theta)$. $\endgroup$ – MInner Sep 16 '17 at 0:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.