How to get probability from the confidence score in SVM

Question

In liblinear library we can get confidence score (the distance between decision hyperplane) in SVM solver for a binary classification problem, but if i want a probability value for membership in any class, which is not directly available in SVM in liblinear, is there any way to transform the confidence score into probability value? My question is just: Can we get a probability value in SVM rather than just 0-1 value? The platt scaling can do this in libsvm, but the platt scaling suffers theoretical issues, is there other way?because I want to calculate the AUC for measuring SVM's classifying performance.

Answer 1

SVMs do not produce probabilities, so you can not obtain a probability score from just an SVM.

If you want a valid probability score, use logistic regression. Since you are using lib linear, the difference in performance between a linear SVM and linear Logistic model is minimal, but the logistic model will actually have real probabilities.

Note, there are various "probability calibration" techniques that can be used, such as Platt's scaling. Some more info in a blog post of mine here.

Answer 2

You're reasoning from a false premise. To compute AUC from the signed distances, all you need to do is compute TPR and FPR at each value of the signed distances for the observations in the hold-out set. ROC curves don't need probabilities as inputs. Actually, it's provable that ROC curves and AUC don't depend on having probabilities as inputs: if all positives have prediction 100 and all negatives -100, then the AUC is 1 because the probability that a randomly-selected positive is ranked higher than a randomly-selected negative is 1 -- exploiting the connection to the c-statistic.