I get contradictory information on what the penalty value C does in SVM.

page 346,347 of the following book says, larger C means larger misclassification is allowed and margin will be larger. http://faculty.marshall.usc.edu/gareth-james/ISL/ISLR%20Seventh%20Printing.pdf

same says the below link (in regularization section) https://www.datacamp.com/community/tutorials/svm-classification-scikit-learn-python

However, every other source I read says C in svm class of python stands for penalty and larger C means higher penalty and thereby lower margin. (But going by the above book and the optimization equations, C is the allowance for misclassification and this seems correct - based on the formula).

Can someone please explain why the above two links are contradicting other sources

  • $\begingroup$ I think the first one is wrong. Larger cost means that misclassification is penalized more, which should imply a lower margin. $\endgroup$ – George Jul 31 '19 at 19:22

1.Book ISLR - tuning parameter C is defined as the upper bound of the sum of all slack variables. The larger the C, the larger the slack variables. Higher C means wider margin, also, more tolerance of misclassification.

2.The other source (including Python and other online tutorials) is looking at another forms of optimization. The tuning parameter C is NOT defined as the the upper bound of the sum of all slack variables. On the contrary, larger C means more constraints on the slack variables, and smaller C means less constraints on the slack variables. enter image description here

Small C means the penalty part (the right part) plays little role. We don't need the slack variables to be very small for the minimization process. We have more tolerance of misclassification. The margin will be wide.

Large C means that the penalty parts plays an important role. We want to make the slack variables as small as possible. We won't tolerate misclassification. The margin will be narrow.

| cite | improve this answer | |
  • $\begingroup$ Thank you @Sophie $\endgroup$ – tjt Feb 2 at 1:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.