1
$\begingroup$

Literature and resources say that when the ML log-likelhood does not have a closed form expression, then we can use Newton-Raphson and other optimization techniques. My Question is:

During estimation of the ML estimator of an unknown parameter, upon taking the derivative of the log-likelihood and equating it to zero does not yield the estimator, then can we apply NR and other optimization techniuqes to solve the log-likelihood?

$\endgroup$
3
  • 1
    $\begingroup$ Could you explain in what sense finding the critical points "does not yield the estimator"? $\endgroup$
    – whuber
    Feb 3, 2015 at 21:16
  • $\begingroup$ What I meant was that the parameter becomes zero say all the terms cancel out. $\endgroup$
    – Srishti M
    Feb 3, 2015 at 21:28
  • $\begingroup$ That sounds like a computational error on your part. $\endgroup$
    – whuber
    Feb 3, 2015 at 21:29

1 Answer 1

2
$\begingroup$

As a general principle, pretty much any valid approach for identifying the argmax of a function may be suitable.

There are many situations where calculus is of no direct help in maximizing a likelihood, but a maximum can still be readily identified; there's nothing that gives setting the first derivative equal to zero any kind of 'primacy' or special place in finding the parameter value(s) that maximize log-likelihood. It's simply a convenient tool.

It's very common to use optimization techniques to maximize likelihood; there are a large variety of methods (Newton's method, Fisher scoring, various conjugate gradient-based approaches, steepest descent, Nelder-Mead type (simplex) approaches, and a wide variety of other techniques)

[That said, there's not enough detail in the question to be sure that you're in one of those situations where calculus doesn't get you there.]

$\endgroup$
3
  • $\begingroup$ Thank you for your insights. As pointed out by whuber the reason the expression of the estimator becomes zero can be attributed to error in derivation. I thought maybe MLE does not exist. In that case, I thought to maximize the log likelihood using the optimization techniques which you mentioned. $\endgroup$
    – Srishti M
    Feb 4, 2015 at 1:00
  • $\begingroup$ Based on the comments & your reply, I can infer that under these sitautions one can use optimization if (a) the derivative of log likelihood is too complex (b) no closed form solution (c) optimize the solution to avoid local estimated point. Otherwise, if expression is coming zero then there is other problem. Am I correct? $\endgroup$
    – Srishti M
    Feb 4, 2015 at 1:01
  • $\begingroup$ 1. one can use optimization methods even if it's not too complex, and even if it does have a closed form solution. $\:$ 2. I don't know for certain if there's another problem or not (I suspect so); it might indicate an algebraic error [or depending on exactly what the precise issue is, it might indicate a need for say l'Hôpital's rule or it might indicate something else (it might indicate something about the problem that you neglected to mention, for example). You should show the actual formula and the calculation for a more informed comment.] $\endgroup$
    – Glen_b
    Feb 4, 2015 at 1:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.