3
$\begingroup$

I am getting a strange output from sklearn's LogisticRegression, where my trained model classifies all observations as 1s.

In [1]:
logit = LogisticRegression(C=10e9, random_state=42)
model = logit.fit(X_train, y_train)
classes = model.predict(X_test)
probs = model.predict_proba(X_test)

print np.bincount(classes)

Out [1]: 
[   0 2458]

But look at the predicted probabilities: predicted_probability_histogram

How is this possible?

I know that there is another post on this (here), but it does not answer this question. I understand that my classes are not balanced (this uniform classification goes away when I enter the argument class_weights = balanced).

However, I want to understand why sklearn is classifying predicted probabilities of less than 0.5 as a positive event.

Thoughts?

$\endgroup$
2
  • $\begingroup$ This focuses on sklearn but I think it's really a question about logistic regression so I am voting to leave it open. $\endgroup$
    – Peter Flom
    Jun 30, 2018 at 13:53
  • $\begingroup$ Hi @PeterFlom, I see your point, but I would argue that it is more about what sklearn does with the output than logistic regression per se. The issue and the answer both center on how sklearn's predict_proba returns predictions for both classes. From a regression standpoint, it is curious that the model only predicts 1s, but that is an artifact of class imbalance. $\endgroup$
    – NLR
    Jun 30, 2018 at 17:47

1 Answer 1

9
$\begingroup$

Notice how your plot is symmetric? That's because predict_proba has shape (n_samples, n_classes), so half the data you've plotted is redundant with the other half (since $p_i + (1 - p_i) = 1$).

If you look at probs[:,1] by itself I'm sure it will make sense.

$\endgroup$
3
  • $\begingroup$ Ah, I see: predict_proba returns Pr(y=1 | X) and Pr(y=0 |X) for each sample. Makes sense. Thanks @Sycorax! $\endgroup$
    – NLR
    Jun 29, 2018 at 20:30
  • $\begingroup$ If you've found this helpful, please consider upvoting and/or accepting my answer. $\endgroup$
    – Sycorax
    Jun 29, 2018 at 20:32
  • $\begingroup$ I'm a newb on here, so I'm not able to vote yet :/ $\endgroup$
    – NLR
    Jun 29, 2018 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.