1
$\begingroup$

I often use the instance weights with Libsvm for classification problems. http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/#weights_for_data_instances

Does anyone know the details of the algorithm that is implemented when one uses instance weighing in libsvm? The standard SVM model learning algorithm assigns equal weights to all training instances, and thus to the error on the training instances. I believe that the algorithm that Libsvm uses would be different. Upon searching online, I do find some papers that do something similar. For example [1] but I need to confirm with someone who may be sure about this.

Thanks!

ps: i have also posted asked this on stackoverflow. Will update if I get any answer there.

[1] Yang, Xulei, Qing Song, and Yue Wang. "A weighted support vector machine for data classification." International Journal of Pattern Recognition and Artificial Intelligence 21.05 (2007): 961-976.

$\endgroup$
2
  • $\begingroup$ The help asks that you avoid crossposting; third-last paragraph here $\endgroup$
    – Glen_b
    Commented Aug 28, 2014 at 4:19
  • $\begingroup$ The optimization problem is essentially the same. For instance, you can get the effect of $C_x = 2 C_y$ by sampling instance $X$ twice and solving with the classic optimization algorithm. $\endgroup$ Commented Mar 29, 2015 at 8:35

2 Answers 2

2
$\begingroup$

I believe that it's equivalent to the problem formulation in the given paper: $C$ is simply scaled for each instance by $W_i$. See lines 2250-2265 of svm.cpp in the 3.18 release and how it's used in the call to svm_train_one on line 2309.

$\endgroup$
1
$\begingroup$

For the actual support vectors, the weight obviously should not make a difference.

It does however make a difference for which solution is optimal - essentially, if you have "k" times the same point, you want to scale the costs by "k". AFAICT, this should be trivially possible in SVM, too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.