0
$\begingroup$

I have some experience with building classification models in R but it's my first time with Python's sklearn.

So the the problem is: when fitting logistic regression model in R, I have immediate access to predicted class probabilities (model$fitted.values), so I can set my threshold (different than 0.5) in order to maximize some measure.

But in sklearn after fitting I can't find a way to access probabilities. Is it possible? There is a method predict_proba(), but...as the name suggests, it is prediction. So in order to get probabilities, should I 'artificially' do the following procedure?

model = sklearn.linear_model.LogisticRegression()
model.fit(train_X, train_y)
probs = model.predict_proba(train_X)

Does it make any sense? Or is there some different method to obtain it?

$\endgroup$
2
$\begingroup$

It is correct to use the method predict_proba to get the estimated probabilities. Just to make it a bit clearer, consider the example given here (http://scikit-learn.org/stable/auto_examples/linear_model/plot_iris_logistic.html).

Below I copy just the relevant lines, so that anyone can copy & paste it, and reproduce what I say next,

import numpy as np
from sklearn import linear_model, datasets
iris = datasets.load_iris()
X = iris.data[:, :2]  # we only take the first two features.
Y = iris.target
h = .02  # step size in the mesh
logreg = linear_model.LogisticRegression(C=1e5)
logreg.fit(X, Y)

In the iris dataset you have three classes (0, 1, 2 = ['setosa', 'versicolor', 'virginica']), contained in Y. When you call,

logreg.predict_proba(X)

you get an array of arrays of probabilities, each element being p(class|x),

array([[  9.05823905e-01,   6.81672013e-02,   2.60088939e-02],
       [  7.64631786e-01,   2.16376590e-01,   1.89916235e-02],
       [  8.46908157e-01,   1.42190177e-01,   1.09016662e-02],
       [  8.15654921e-01,   1.75608861e-01,   8.73621791e-03],
       ...

The other alternatives are predict_log_proba() to get log P(class|x) and predict() to get the class with highest probability among the three for a given sample.

$\endgroup$
1
$\begingroup$

Fitting the model just optimizes coeffecients and intercept. Introducing special thresholds only affects in the proportion of false positives/false negatives (and thus in precision/recall tradeoff), but it is not the parameter of the LR model.

You predict the probabilities like you described and apply a threshold to get actual class predictions, for example class_predictions = preds > 0.6

Also check this

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.