C penalty in SVM - larger C increases the margin or reduces the margin?

Question

I get contradictory information on what the penalty value C does in SVM.

page 346,347 of the following book says, larger C means larger misclassification is allowed and margin will be larger. http://faculty.marshall.usc.edu/gareth-james/ISL/ISLR%20Seventh%20Printing.pdf

same says the below link (in regularization section) https://www.datacamp.com/community/tutorials/svm-classification-scikit-learn-python

However, every other source I read says C in svm class of python stands for penalty and larger C means higher penalty and thereby lower margin. (But going by the above book and the optimization equations, C is the allowance for misclassification and this seems correct - based on the formula).

Can someone please explain why the above two links are contradicting other sources

Answer 1

1.Book ISLR - tuning parameter C is defined as the upper bound of the sum of all slack variables. The larger the C, the larger the slack variables. Higher C means wider margin, also, more tolerance of misclassification.

2.The other source (including Python and other online tutorials) is looking at another forms of optimization. The tuning parameter C is NOT defined as the the upper bound of the sum of all slack variables. On the contrary, larger C means more constraints on the slack variables, and smaller C means less constraints on the slack variables.

Small C means the penalty part (the right part) plays little role. We don't need the slack variables to be very small for the minimization process. We have more tolerance of misclassification. The margin will be wide.

Large C means that the penalty parts plays an important role. We want to make the slack variables as small as possible. We won't tolerate misclassification. The margin will be narrow.