In Pattern Recognition and Machine Learning Section 7.1:

enter image description here enter image description here enter image description here

Based on what I understood so far, the slack variable $\xi$ is defined as $max(0, 1-t_ny(x_n))$ and it's associated with the hinge loss.

However it seems to me that the two constraints $t_ny(x_n)\geq1-\xi_n$ and $\xi_n\geq0$ are just two properties of $\xi$ according to on how it is defined, and without them it is still a valid optimization problem to solve (hinge loss + regularizer).

Why do we want to use them as the constraints again?
Or the slack variable is not explicitly defined as $max(0, 1-t_ny(x_n))$ but is only defined by the constraints?

Please correct me where I'm wrong.


So I have found in another book that, introducing the slack variable

$\min_{w,b,\xi} \frac{1}{2}||w||^2+C\sum^N_{i=1}\xi_i$


$t_iy(x_i)\geq1-\xi_i$ and $\xi_i\geq0$

is essentially a rewrite of the hinge+regularizer loss,

$\min_{w,b} \frac{1}{2}||w||^2+C\sum^N_{i=1}max(0, 1-t_iy(x_i))$.

So the slack variable $\xi$ is implicitly defined as $max(0, 1-t_iy(x_i))$ by the constraints.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.