2
$\begingroup$

I think I understand likelihood, and also log-likelihood as well. After reading about log-likelihood in various sources, I thought that the purpose of taking the logarithm of likelihood was all about computational convenience since adding up too small values is easier than multiplying them.

However, my instructor claims that logarithm in likelihood is not used only for the computational purpose. He tried to explain very patiently but I missed that part somehow. So, I am curious about its other purposes, what are they?

edit: after transforming a likelihood equation into a logarithm, we can also take its derivative and equal it to zero to find the maximum value to make observed value most likely. This is not my question since it appears to me this is also about computation or optimizing.

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ Possible duplicate of Why to optimize max log probability instead of probability $\endgroup$
    – Tim
    Commented Oct 13, 2016 at 11:26
  • $\begingroup$ but as far as I can understand, all the answer are from the perspective of either computational or optimization. I wonder is there any reason rather than these? maybe there is not? $\endgroup$
    – Muatik
    Commented Oct 13, 2016 at 11:49
  • 2
    $\begingroup$ Outlined in bold in my answer. The derivative of the natural logarithm (there's the need for the log) of the likelihood function gives you the score. The second derivative gives you the Information. Information being a non-computational consideration but one of 'trust' in the estimated parameter value. High curvature in the region around the MLE means high amount of trust in the estimated value. Thus, using the log-likelihood leads us to a mathematical expression that gives us a handle of how well we're estimating the parameter of interest. $\endgroup$
    – Beyer
    Commented Oct 13, 2016 at 11:54
  • $\begingroup$ I've pasted the comment into the actual answer where it really belongs. $\endgroup$
    – Beyer
    Commented Oct 13, 2016 at 12:00

1 Answer 1

Reset to default
3
$\begingroup$

One reason is for ease of optimizing: Why to optimize max log probability instead of probability

Another reason relates to better handling of very small values (See Luca's answer): I am wondering why we use negative (log) likelihood sometimes?

And finally (Courtesy of Whuber's informative comment here: Why apply log to likelihood?) because of the Fisher Information: (From Wikipedia: https://en.wikipedia.org/wiki/Fisher_information)

The Fisher information is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter θ upon which the probability of X depends. The probability function for X, which is also the likelihood function for θ, is a function f(X; θ); it is the probability mass (or probability density) of the random variable X conditional on the value of θ. The partial derivative with respect to θ of the natural logarithm of the likelihood function is called the score.

Under certain regularity conditions,[4] it can be shown that the first moment of the score (that is, its expected value) is 0... <snip> The second moment is called the Fisher information:

Moved from comment to answer: The derivative of the natural logarithm (there's the need for the log) of the likelihood function gives you the score. The second derivative gives you the Information. Information being a non-computational consideration but one of 'trust' in the estimated parameter value.

High curvature in the region around the MLE means high amount of trust in the estimated value. Thus, using the log-likelihood leads us to a mathematical expression that gives us a handle of how well we're estimating the parameter of interest.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.