# L2 SVM (squared hinge) theory

The linear L2 SVM can be intuitively understood as

$$\text{minimize } f(\boldsymbol{w}) = \frac{1}{2} \Vert\boldsymbol{w}\Vert^2_2 + C \sum_{i=1}^m \xi_i^2 \tag{1}$$ where $\xi_i = \text{max}(0, 1 - y_i\boldsymbol{w}^T\boldsymbol{x}_i)$.

However, in papers, I often find it presented instead as (1) s.t. $\xi_i \geq 1 - y_i\boldsymbol{w}^T\boldsymbol{x}_i$.

What are the mathematical conditions that allow the equality constraint $\xi_i = \text{max}(0, 1 - y_i\boldsymbol{w}^T\boldsymbol{x}_i)$ to be recast as the inequality constraint $\xi_i \geq 1 - y_i\boldsymbol{w}^T\boldsymbol{x}_i$?

[To be clear, the optimization problem with the inequality constraint is an optimization with respect to $w$ as well as $\xi_1, \ldots, \xi_m$.]
Consider the version with the inequality constraint. Suppose $w$ is fixed, and we want to find the best $\xi_i$; this is simply minimizing $\xi_i^2$ under the constraint $\xi_i \ge 1 - y_i w^\top x_i$. Do you see that the best $\xi_i$ is $\xi_i = \max(0, 1- y_i w^\top x_i)$?
• @TimMak I don't think anything special is going on here beyond the fact that the solution of optimizing $\xi_i$ with the inequality constraint is precisely the equality constraint. – angryavian May 14 '18 at 15:57