1
$\begingroup$

I have seen two different ways to formulate the SVM optimization but I was not sure what the difference was between them or if there was any difference.

First formulation:

$$min \frac{1}{2}||\theta||^2 + C \sum^n_{i=1}\xi_i $$ $$\text{s.t. }\ \ y^{(i)}(\theta \cdot x^{(i)} +\theta_0) \geq 1 - \xi_i$$

Second formulation:

$$min \frac{\lambda}{2}||\theta||^2 + \frac{1}{n} \sum^n_{i=1}\xi_i $$ $$\text{s.t. }\ \ y^{(i)}(\theta \cdot x^{(i)} +\theta_0) \geq 1 - \xi_i$$

Are they the same formulation?

The thing that is confusing me is that I think what they mean that:

$$min \frac{1}{2}||\theta||^2 + \frac{1}{\lambda} \sum^n_{i=1}\xi_i $$

where:

$$C = \frac{1}{\lambda}$$

Is the same as:

$$min \frac{\lambda}{2}||\theta||^2 + \sum^n_{i=1}\xi_i $$

Am I right that there should not be and 1/n?

Or does it make a difference?

$\endgroup$
2
$\begingroup$

I doesn't make the difference given $C, \lambda >0$ because in both cases minimum will be achieved at same $\theta, \xi_i$. In original paper on soft-margin SVM there were no $\frac{1}{n}$, although it still won't matter because that coefficient is non-negative.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.