# sklearn LogisticRegression only predicts 1, but predict_proba has many values

I am getting a strange output from sklearn's LogisticRegression, where my trained model classifies all observations as 1s.

In :
logit = LogisticRegression(C=10e9, random_state=42)
model = logit.fit(X_train, y_train)
classes = model.predict(X_test)
probs = model.predict_proba(X_test)

print np.bincount(classes)

Out :
[   0 2458]


But look at the predicted probabilities: How is this possible?

I know that there is another post on this (here), but it does not answer this question. I understand that my classes are not balanced (this uniform classification goes away when I enter the argument class_weights = balanced).

However, I want to understand why sklearn is classifying predicted probabilities of less than 0.5 as a positive event.

Thoughts?

• This focuses on sklearn but I think it's really a question about logistic regression so I am voting to leave it open. Jun 30, 2018 at 13:53
• Hi @PeterFlom, I see your point, but I would argue that it is more about what sklearn does with the output than logistic regression per se. The issue and the answer both center on how sklearn's predict_proba returns predictions for both classes. From a regression standpoint, it is curious that the model only predicts 1s, but that is an artifact of class imbalance.
– NLR
Jun 30, 2018 at 17:47

Notice how your plot is symmetric? That's because predict_proba has shape (n_samples, n_classes), so half the data you've plotted is redundant with the other half (since $p_i + (1 - p_i) = 1$).

If you look at probs[:,1] by itself I'm sure it will make sense.

• Ah, I see: predict_proba returns Pr(y=1 | X) and Pr(y=0 |X) for each sample. Makes sense. Thanks @Sycorax!
– NLR
Jun 29, 2018 at 20:30