# The Cost Parameter for Support Vector Machines

I am now learning about SVMs and I learned that "cost" is one of the most important tuning parameters for building the best performing model with SVMs. But i found it very hard for me to understand the concept of "cost" because it is generally defined as "the price for misclassifications". Although i can do SVMs in R and have built some fair models, it would not do me any harm if i understand the logic behind the parameter of "cost". Thanks Felix

• SVM objective has two terms: 1) a regularization term (e.g. $||w||^2$), and 2) a loss term $\sum_i \xi_i$ (see @Ben DAI's answer for more details). The hyperparameter $C$ provides a way to balance these two terms. – Sobi Dec 11 '15 at 15:45

$$y_i(\mathbf{w}\cdot\mathbf{x_i} - b) \ge 1 \quad 1 \le i \le n. \quad\quad(2)$$
to $$y_i(\mathbf{w}\cdot\mathbf{x_i} - b) \ge 1 - \xi_i \quad 1 \le i \le n. \quad\quad(2)$$ The slack variable $\xi$ able to states the "misclassification".
However, we can not allow too much "misclassification". In this light we rewrite the objective function, $$\arg\min_{\mathbf{w},\mathbf{\xi}, b } \left\{\frac{1}{2} \|\mathbf{w}\|^2 + C \sum_{i=1}^n \xi_i \right\}$$ The tuning parameter $C$ which you claim "the price of the misclassification" is exactly the weight for penalizing the "soft margin".
There are many methods or routines to find the optimal parameter $C$ for specific training data, such as Cross Validation in LiblineaR.