Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am using LibLinear for a document classification task, in which I would like to calculate the probability of correctness for each prediction. In fact, in the LibLinear, it does provide probability output for logistic regression, but not for default support vector classification task. Furthermore, based on 10-fold cross validation, the logistic regression is nearly 10% worse than support vector classification.

So can anyone tell me, if I continue to use the support vector classification for the solution, is there a method to calculate the probability separately from the program? I greatly appreciate your help !

share|improve this question

2 Answers 2

You can use a sigmoid function $f(d) = \frac{1}{1 + e^{-\alpha(d-\beta)}}$ to convert your SVM decision value $d = (w, x) + b$ into a number between 0 and 1 which can be treated as probability. You can adjust parameters $\alpha$ and $\beta$ depending on your data.

For more elaborate approaches, see these papers:

  • B.Zadrozny, C. Elkan, Transforming classifier scores into accurate multiclass probability estimates.
  • J.Drish, Obtaining calibrated probability estimates from Support Vector Machines.
share|improve this answer

At least in R, only two algorithms provide the probabilities in LiblineaR interface.

Here is the FAQ of the actual library:

Q: How do I choose the solver? Should I use logistic regression or linear SVM? How about L1/L2 regularization?
Generally we recommend linear SVM as its training is faster and the accuracy is competitive. However, if you would like to have probability outputs, you may consider logistic regression.

Moreover, try L2 regularization first unless you need a sparse model. For most cases, L1 regularization does not give higher accuracy but may be slightly slower in training.

Among L2-regularized SVM solvers, try the default one (L2-loss SVC dual) first. If it is too slow, use the option -s 2 to solve the primal problem.

It seems the svm algorithms don't provide probabilities as output.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.