0
$\begingroup$

When using logistic regression in Python's scikit-learn, one may handle multiclass problems even with binary logistic regression. If there are $K$ classes, then coefficients (i.e. weights and biases) for $K$ logistic functions will be produced. But this is using a 'one vs. rest' approach, and the probabilities from the individual logistic functions won't necessarily add up to 1 since this is binary logistic regression. Therefore, when using predict_proba with sklearn's logistic regression, how are probabilities handled in multiclass problems?

I've investigated this and it appears to be similar to applying a softmax function to the individual probabilities of the $K$ logistic functions, but this is not exactly correct. I also do not see explicit mention of this in the documentation.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

It appears that, they just apply simple normalization (i.e. divide by the sum of the probabilities) or softmax when multi_class option is set to ovr or multinomial respectively.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I had tested so many alternatives...yet somehow missed the simplest approach. Thank you. Although, is there any meaningful justification in applying the simple normalization they've used for the case of ovr? Since these are essentially from separately trained binary logistic classifiers, there doesn't appear to be any reason why the probabilities from the different models are valid for direct comparison. $\endgroup$
    – Mathews24
    Nov 14, 2018 at 1:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.