rb-libsvm - getting confidence of a prediction

Question

I'm using rb-libsvm and the RBF kernel to make classifications. svm.predict(measurements) returns either -1.0 or 1.0. Is there a way to get a confidence for this classification? I am interested in throwing out low-confidence classifications and tweaking precision/recall.

Answer 1

There's a method called predict_probability which returns a distribution for the prediction

Answer 2

I don't have much experience with the specific implementation of SVM you are using, however in general there are a range of methods for calibrating the output of an SVM classifier into probabilities. For example, the R library 'kernlab' has an option to fit a probability model to the distances of points from the support vectors via a sigmoid function. This can then be used to predict probabilities instead of just the best guess classes.

An interesting discussion of SVM calibration and optimisation of F1 scores can be found here. In general, the key concept to search for is calibration of classifier output.