In article LIBSVM: A Library for Support Vector Machines there is written, than C-SVC uses loss function:

$$ \frac{1}{2}w^Tw+C\sum\limits_{i=1}^l\xi_i$$

OK, I know, what is $w^Tw$.

But what is $\xi_i$? I know, that it is somehow connected with misclassifications, but is it calculated exactly?

P.S. I don't use any non-linear kernels.


$\xi_i$ are the slack variables. They are typically nonzero when the 2-class data is non-separable. We are trying the minimize the slack as much as possible (by minimizing their sum, since they are non-negative) along with maximizing the margin ($w^Tw$) term.

Exact calculation: Well, if the convex program has been solved to optimality without any optimization error, then yes, they are calculated exactly.

  • $\begingroup$ I know, that they are typically nonzero when the 2-class data is non-separable. I know, that we are trying to minimize their sum. But I don't know, what is the loss function they are calculated with. Is it a step function or hinge loss or something else? $\endgroup$
    – Felix
    Oct 14 '13 at 20:59
  • 1
    $\begingroup$ Yes, it is the hinge loss. The hinge loss has been removed from the objective and made into a bunch of constraints (their number equalt to the number of examples, $l$ in your notation). In particular, $\max[0,1-y_iw^Tx_i]$ is the loss on example $i$. $\endgroup$ Oct 14 '13 at 21:14
  • $\begingroup$ @Theja can you explain how wTw term is same as maximum margin? $\endgroup$ Feb 19 '16 at 6:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.