2
$\begingroup$

From sklearn's documentation here:

The decision_function method of SVC and NuSVC gives per-class scores for each sample (or a single score per sample in the binary case). When the constructor option probability is set to True, class membership probability estimates (from the methods predict_proba and predict_log_proba) are enabled. In the binary case, the probabilities are calibrated using Platt scaling [9]: logistic regression on the SVM’s scores, fit by an additional cross-validation on the training data. In the multiclass case, this is extended as per [10].

I'm confused on the procedure sklearn uses. From the reference on Platt Scaling, my understanding of the procedure is:

  1. Run SVM on the entire training set as normal

  2. Run SVM with k-fold cross validation on the training set

  3. Train one logistic regression model using SVM scores for the entire training set (by combining them from each hold out set)

  4. At inference, data is first ran through the SVM model from 1 then through the logistic regression model

Is the above correct? In the sklearn docs, it seems to read that cross validation is applied to the logistic regression portion?

$\endgroup$

1 Answer 1

1
$\begingroup$

Inspecting the docs for CalibratedClassifierCV may offer more clarity (emphasis my own):

This class uses cross-validation to both estimate the parameters of a classifier and subsequently calibrate a classifier. With default ensemble=True, for each cv split it fits a copy of the base estimator to the training subset, and calibrates it using the testing subset. For prediction, predicted probabilities are averaged across these individual calibrated classifiers. When ensemble=False, cross-validation is used to obtain unbiased predictions, via cross_val_predict, which are then used for calibration. For prediction, the base estimator, trained using all the data, is used. This is the method implemented when probabilities=True for sklearn.svm estimators.

The "for calibration" part is presumably shorthand for your step 3. So your procedure is correct.

Verifying this fact in the code is tricky. I could only verify that, for binary classification with probability=True, exactly two parameters are learned: one for the slope in logistic regression, and another for the intercept. See the properties probA_ and probB_, which must be non-empty to run predict_proba.

I agree that this part of the documentation you linked should be re-worded to

the probabilities are calibrated using Platt scaling [9]: logistic regression on the SVM’s scores, fit by an additional cross-validation on the training data. obtained via cross_val_predict.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.