Typical support vector classifier uses the following optimization procedure:

$$\min ||w||^2 + C\sum_{i=1}^N \zeta_i$$
$$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$
$$\zeta_i \geq 0$$

This hinge loss setup slightly penalizes the correctly classified data points within the margin. Now if we gently modify the constraint the result will be a learning machinery with a regularized perceptron loss.

$$y_i(w^Tx_i+b) \geq - \zeta_i$$

[![enter image description here][1]][1]

I understand there are historical reasons things are the way they are (maximizing margin rhetoric, etc.) But is there a particular theoretical reason for not implementing a support vector classifier in this manner? What will be the pros and cons?


  [1]: https://i.sstatic.net/lNgJE.png