I am studying SVM from Andrew ng machine learning notes. I don't fully understand the optimization problem for svm that is stated in the notes. So we have optimization problem

$$\max_{\gamma, w, b}\gamma$$ s.t. : $$y^{(i)}(w^Tx^{(i)}+b)\geq\gamma, i=1,\dots m,\\||w||=1.$$

I get a little bit confused here, as i don't see why we should need this first constraint. Isn't it enough only to maximize $\gamma$ and in that case we already have the maximal margin hyperplane where observations are at least $\gamma$ away from the decision boundry (because $\gamma$ is already defined this way)?

I think I don't understand something simple here. If you have any explanation for this I would appreciate it very much.


1 Answer 1


(because $\gamma$ is already defined this way)?

But how does the optimization problem know that $\gamma$ is defined in such way ? if you remove that constraint, your optimization problem becomes:

$$ \max_{\gamma, b, w} \gamma \\ s.t. : ||w|| = 1$$

The optimization problem will just indefinitely increase $\gamma$, as is not restricted in any way. You have to enforce the way gamma is defined so that the optimization problem has solutions that respect that definition.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.