1
$\begingroup$

The common formulation of the SVM problem is

$$\min_{\theta, \theta_0}\frac{1}{2}||\theta||^2$$ $$\text{ subject to: } y^{(t)}(\theta \cdot x^{(t)} + \theta_0) \geq 1, \ t=1,...,n,$$

However, it seems to be unnatural that $\theta_0$ (something we try to optimize) is a part of the constraint, but it doesn't appear in the objective.

Is this natural?

$\endgroup$
1
$\begingroup$

Yes, because SVM is trying to maximize the margin between supporting planes. The margin is $2\over||\theta||$, which is the distance between the supporting hyperplanes. Distance between parallel hyperplanes is $\frac{\text{bias difference}}{||\text{normal vector}||}$, where normal vector is $\theta$ here. For classes $1,-1$ the bias terms are $1-\theta_0$ and $-1-\theta_0$, so their difference is $2$. Maximizing this term is equivalent to minimizing $\frac{||\theta||^2}{2}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.