4
$\begingroup$

I am trying to implement a logistic regression function in c++, and not sure what algorithm to use. So far I have heard of these:

  1. Newton-Raphson
  2. IRLS
  3. Gradient descent

Are there other algorithms available? What are their pros and cons? Is there an algorithm generally recognized as superior to the others?

$\endgroup$
  • 3
    $\begingroup$ Pretty much any decent optimization algorithm can work under at least some circumstances. Fisher scoring is very widely used and is convenient. $\endgroup$ – Glen_b Mar 31 '14 at 9:17
  • 2
    $\begingroup$ And if you incorporate step-halving into the algorithms they usually work well. $\endgroup$ – Frank Harrell Mar 31 '14 at 12:29
  • 2
    $\begingroup$ In the context of logistic regression, Newton's method reduces to iteratively reweighted least squares. Working this out is an excellent exercise. $\endgroup$ – Matthew Drury Jun 5 '15 at 3:28
3
$\begingroup$

Partially answered in comments:

Pretty much any decent optimization algorithm can work under at least some circumstances. Fisher scoring is very widely used and is convenient. – Glen_b

And if you incorporate step-halving into the algorithms they usually work well. – Frank Harrell

Much used is iteratively reweighted least squares (IRLS), which is a reformulation of the Newton method. See here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.