# Support Vector machine : a simple question

I think that a formulation of SVM for points x with label y is :

\begin{align} \arg\min_{\substack{u,w,b}} \frac{1}{2} \cdot |w|^2 + C \cdot \sum_{i} u_i \\ s.t.\ \ y_i\cdot (w \cdot x_i + b) &\geq 1-u_i \\\\ u_i \geq 0 \\ \end{align}

In that formulation, if we take C = 0, what prevents $u_i$ to go to infinity, so that the two constraints are always satisfied ?

• my guess is that C has to be non zero, but I cant quite prove it. Apr 7, 2013 at 17:01
• If you set $C$ to zero, you are saying you want to totally ignore violation of the margin constraint. If you actually want to do that, you shouldn't be using the soft margin form in the first place. Apr 8, 2013 at 0:10
• indeed. I see that if we set C to 0 we must remove the u_i, but I thought there'd be some deeper insight . Apr 8, 2013 at 10:15

I don't think the $u_i$s will go to infinity. If $C$ is set to zero this effectively disables the inequality constraints, so the optimisation problem is just to minimise the squared norm of the weights, which has a trivial solution at $w = 0$. Substituting this into $y_i(w \cdot x_i + b) \geq 1 - u_i$ gives $y_i b \geq 1 - u_i$ which is satisfied by $u_i = 1 - y_i b$. This means there will be a solution where the $u_i$ are finite, provided a sensible choice is made for $b$. Whether the software actually finds this solution is another matter though; however as setting C=0 is effectively telling the SVM to completely ignore the training data, it is perhaps not too surprising that the programmer didn't consider this!
Actually, the slack variable $u_i$ means the tolerance of inconsistent labeling with $y_i$ by the linear function $(wx_i +b)$. The factor $C$ "adjusts" the weight of total tolerance. Therefore, setting $C$ to zero might be problematic in the case that the set $\{x_i,y_i\}$ is not linearly separable!