Would the support vectors in SVM algorithm change with scaling of the functional margin?

Question

Would the Support Vectors in the SVM algorithm change every time that I change the functional margin ? The optimization objective in the SVM algorithm is this

The rest of the SVM optimization algorithm is then derived by setting i.e. $\hat{\gamma}=1$. Does the Support Vectors derived depend on $\gamma$, $w$ or $b$ in any manner?

Answer 1

The original answer is here courtesy Arun Iyer.

The entire algorithm is then derived by setting i.e. Gamma(hat) to 1

That is not exactly true, although that’s certainly one way to think about it. The algorithm proceeds by replacing w and b with new variables $w=\hat{\gamma}w'$ and $b=\hat{\gamma}b'$. You will notice that when you do these substitutions, $\hat{\gamma}$ will disappear.

So, in this new formulation you are solving for $w'$ and $b'$ instead of $w$ and $b$. Now, given this information, you can make two observations:

1. You can obtain the solution for the original formulation for any $\hat{\gamma}$ from the new formulation by scaling $w'$ and $b'$ with $\hat{\gamma}$ to get $w$ and $b$.
2. So, we can reason about the solution to the original problem and find that the solution to the original problem must be $\hat{\gamma}=\frac{1}{||w'||},w=\frac{w'}{||w'||}, b=\frac{b'}{||w'||}$

Now, given all this information, can you reason about the support vectors since you know the forms of $w'$ and $b'$ from the KKT conditions?

The answer is that Support Vectors will change but only in magnitude.