I'm trying to understand the implementation of scikit-learn's Logistic Regression. I am new to the framework, and have only a basic understanding of logistic regression.


Specifically, I wish to understand the Solver parameter. I was following another tutorial that uses logistic regression and suggests that fitting the model was done by Maximum Likelihood Estimation.

Now the API from scikit-learn does not mention using the MLE, but does mention a Solver parameter with the following types: solver : {‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’}, default: ‘liblinear’

1) Is MLE and the Solver parameter the same thing or anyway related?

2) If different, can I get a good description of what each does in regards to Logistic Regression. It is my understanding that MLE finds the coefficients (makes sense). The only documentation for the Solver parameter is: Algorithm to use in the optimization problem.

3) The default liblinear is what is used in the already mentioned tutorial. Is this an MLE implementation?

  • $\begingroup$ Solver refers to the optimization method to use to find the optimum of the objective function. MLE defines the objective function under optimization. $\endgroup$ – Sycorax Oct 24 '16 at 19:29
  • $\begingroup$ Thanks for the input! So one additional question: Does Logistic Regression only use MLE as its objective function or are there other objective functions? $\endgroup$ – user3547551 Oct 24 '16 at 19:53
  • 1
    $\begingroup$ Facetiously, you could also use cross-entropy, which is identical to a rescaled MLE with a reversed sign. In this case, the objective is minimized. There are lots of classification loss functions (hinge, 0-1), but I think logistic regression is understood to imply MLE or equivalent cross-entropy. $\endgroup$ – Sycorax Oct 24 '16 at 21:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.