Interpretation of C hyperparameter in support vector classifier? I found a contradiction between inteprpetation given by university book "Introduction to Statistical Learning" and all the other web sources. In the first one it is said that the larger is C the wider is the margin, while everywhere in the web (DataCamp, Towards Data Science, etc) it is said that the larger is C the smaller is the margin. Which of the two interpretations is correct? And which one is used by R and Python?
 A: The two use different formulations of the same penalty parameter. In Intro to 
Stat Learning, it is formulated as
$$
\max_{\beta_0, \beta_1, ... \beta_p, \epsilon_1, ... \epsilon_n, M} M \\
\text{subject to} \\
\sum_{j=1}^p \beta_j^2=1, \\
y_i(\beta_0 + \beta_1 x_{i1} + ... + \beta_px_{ip}) \geq M(1-\epsilon_i),\\
\epsilon_i \geq 0,\\
\sum_{i=1}^n \epsilon_i \le C
$$
while in hackerearth it is formulated as
$$
\min_{\vec{w}, b} \frac{||\vec{w}||^2}{2} + C \sum_{i=1}^n \zeta_i \\
\text{subject to} \\
y_i (\vec{w}.\vec{x}+b) \geq 1 - \zeta_i \\ \text{for all} \ i \in \{1, ..., n\},\\
\zeta_i \geq 0\\
\\
$$
Here $b$ is $\beta_0$, $\vec{w}$ is the vector of all other $\beta_i$. They 
state that minimising the L1 norm of $\vec{w}$ is equivalent to maximising 
the margin (I'm not familiar with SVMs but I can assume this is the case).
A penalty factor applied to the sum of $\zeta_i$ is equivalent to a constraint on the sum of $\epsilon_i$. If we take the extreme cases:
When $C=0$, $C \sum_i^n \zeta_i = 0$, therefore misclassification does not affect the optimisation problem.
If we constrain $\sum_i^n \epsilon_i \le 0$, we disallow misclassification since
any $\epsilon_i > 0$ violates the constraint.
When $C=\infty$, $C \sum_i^n \zeta_i = \infty$ if any $\epsilon_i > 0$, therefore any misclassification makes a solution invalid.
If we constrain $\sum_i^n \epsilon_i \le \infty$, we do not penalise misclassification.
Therefore these two books describe the exact same setup, they just use different
formulations for the penalty parameter. This is possibly just a
terminology/formulation difference arising from differences in background 
(machine learning/ML versus statistics); I would check which version 
an implementation was using before applying it of course. 
This is not totally uncommon with algorithms on the boundary of different fields. For example, in elastic net regression, $\alpha$ typically controls the mix of 
L1 and L2 penalty applied while $\lambda$ controls the strength of the penalty.
In sklearn, $\alpha$ controls the strength of the penalty.
