Would the Support Vectors in the SVM algorithm change every time that I change the functional margin ? The optimization objective in the SVM algorithm is this - enter image description here

The rest of the SVM optimization algorithm is then derived by setting Gamma hat i.e. Gamma (hat) to 1. Does the Support Vectors derived depend on Gamma, w or b in any manner ?

  • $\begingroup$ The complete notes and derivation is here. $\endgroup$ – Sud K Jan 4 '17 at 15:36

The original answer is here courtesy Arun Iyer.

The entire algorithm is then derived by setting i.e. Gamma(hat) to 1

That is not exactly true, although that’s certainly one way to think about it. The algorithm proceeds by replacing w and b with new variables w=γ^w′ and b=γ^b′. You will notice that when you do these substitutions, γ^

will disappear.

So, in this new formulation you are solving for w’ and b’ instead of w and b. Now, given this information, you can make two observations:

You can obtain the solution for the original formulation for any γ^

from the new formulation by scaling w’ and b’ with γ^ to get w and b. So, we can reason about the solution to the original problem and find that the solution to the original problem must be γ^=1∥w′∥,w=w′∥w′∥,b=b′∥w′∥


Now, given all this information, can you reason about the support vectors since you know the forms of w’ and b’ from the KKT conditions?

The answer is that Support Vectors will change but only in magnitude.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.