# Different prediction score for two SVM-based classifiers

As a validation study, I use two libsvm-based svm classifier against the same data set. One classifier is libsvm implementation in Rapidminer. Another classifier is Libsvm itself. Both of them assume the same parameter setting. However, the prediction score of these two classifiers are different. What might be reason to cause this kind of difference?

-
By prediction score you mean $f(x)=(w,x)+b$? What about data normalization, is it used/not used in both cases? What about default parameters of the functions that you are calling, do they have the same values in both cases? –  Leo Sep 28 '12 at 0:15

All:

• Rapidminer
• WEKA
• milions of more

use the libsvm library, which are the same implementations. The only actual difference is how they use it. As underlying code of libsvm is just a numerical optimizator, there is a number of thing that can influence the quality of resulting classifier.

First of all, libsvm requires definition of aa eps parameter that is used in the optimisation process as a threshold value - this is the most common difference between libraries. The second thing is use (or lack of it) of shrinkage heuristics, which guides the optimisation process, and with previously mentioned eps variable can lead to different results.

Those were "deep" reasons for such behaviour. The most simple explanation is... data preprocessing - SVM does not work well on the raw data (which have large disproportions in the input data dimensions), there are at least few possible ways of dealing with it:

• squashing the data linearly to the [-1,1] dimension-wise
• transform the data through C^(-1/2) where C is adata covariance matrix

and again - each "meta library" can use its own method of such preprocessing. And the way it is done greatly influences the results.

-

I know this is a late answer to your question, but I've recently come across a similar situation in my own research. Just out of curiosity, I did a parameter optimization experiment for the c-parameter in a linear svm, in which I wanted to compare the performance of svmlight and the Weka svm implementation. For both approaches, I was classifying the same textual data in 5x2 cross-validation, using binary feature modeling, and bag-of-words unigrams. My findings surprised me: svmlight consistently performed at least 0.10 better (in terms of area under the curve), compared to Weka! I bring this up here, because I think your question raises an interesting point that people don't always think about: not all implementations of svm are the same, and perhaps they should not be treated as such! I think a lot of people will try out the Weka svm on their data (either because it Weka is commonly used, or because it's easier than using a specific implementation like svmlight or libsvm), and falsely conclude, "svms are not good for my data." As to why this is, I am as yet unclear. Perhaps there are differences in the quality of the code, or the optimization routines used in each? I'm going to look into this further!

-
The outcome of SVM training is not subject to change for a given problem. If the training problems for two solvers, e.g. svmlight and libsvm, are exactly the same, their respective solutions should be too. The solution to an SVM training problem is a global optimum and is therefore unique. If you get different solutions, there must be something else happening to the data. –  Marc Claesen Aug 9 '13 at 8:08