For reasons owing to mathematical convenience, when finding MLEs (maximum likelihood estimates), it is often the log-likelihood function---as opposed to the standard likelihood function---which is maximised.

From what I've gathered, this approach is deemed valid as a result of the monotonically increasing nature of the (natural) logarithm function.

My understanding of a monotonically increasing function is: for all $x$ and $y$ (defined on a subset of the reals), if $x \leq y$ then $f(x) \leq f(y)$.

This, however, does not appear to be the case for all log-likelihood functions; for example: for the log-likelihood $\text{Gamma}(3, 5)$ function, if $x = 0.15$ and $y = 0.46$, then $f(x) = -0.59$ and $f(y) = -1.19$.

Clearly, I've misunderstood this concept. Fundamentally, I guess I'm asking can somebody (preferably mathematically) demonstrate why:

$$ \hat{\theta} = \text{argmax} \text{ } L(\theta) = \text{argmax} \text{ } \text{log} \text{ } L(\theta) $$

where $\hat{\theta}$ is the MLE for a given likelihood function.

  • 3
    $\begingroup$ The problem is that the justification you use is not the correct one. It is not the likelihood function that must be monotonous but the transformation you apply to it. E.g. the result comes from the fact that for a concave likelihood function, the value of $\theta$ that maximizes $L(\theta)$ is the same as that which maximizes $\log L(\theta)$. This is a property of monotone transformations of concave functions $\endgroup$ – user603 Feb 25 '13 at 17:55
  • 4
    $\begingroup$ @user603 It's actually the most basic property of monotonic transformations, period: it follows directly from their definition and assumes nothing about $L$ itself. After all, if $L(\theta)\gt L(\theta')$ for all $\theta'$ (that is, $\theta$ maximizes $L$) and $x \gt y$ implies $f(x) \gt f(y)$ whenever $x$ and $y$ are both in the domain of $f$ (that is, $f$ is monotonic), then by letting $x=L(\theta)$ and $y=L(\theta')$ it is immediate that $f(L(\theta))\gt f(L(\theta')))$ for all $\theta'$ (that is, $\theta$ maximizes $f\circ L)$. $\endgroup$ – whuber Feb 25 '13 at 18:32
  • 1
    $\begingroup$ Thanks for your assistance, everybody. As stated above, I had been under the impression that the likelihood function itself must be monotonic (not the transformation applied to it) which didn't seem to make sense. As @whuber states, the property follows directly from the definition of monotonic transformations. $\endgroup$ – user9171 Feb 26 '13 at 11:44

This is a direct consequence of properties of monotone (increasing) transformation, and the logarithm is monotone increasing. If there exist a value of $\theta$ that maximizes the likelihood function, that same value of $\theta$ will maximize the log likelihood function. The later is often preferred because it has better numerical properties, so is easier to maximize in practice. That is not the only reason, the log likelihood function arises also much in theory.

| cite | improve this answer | |
  • $\begingroup$ Pretty sure this is answered elsewhere on site. $\endgroup$ – Glen_b Aug 24 '17 at 18:14
  • $\begingroup$ Probably true ... just search $\endgroup$ – kjetil b halvorsen Aug 24 '17 at 18:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.