3
$\begingroup$

I have read and re-read the original paper and articles on Schölkopf's One-class SVM but elements of it still baffles me. The paper defines the cost function as: $$ L = \frac{1}{2}||w^{2}|| + \frac{1}{\nu}\sum_{i}{(\xi_{i} - \rho)} $$

EDIT: The cost function above is wrong, the correct one reads: $$ L = \frac{1}{2}||w^{2}|| - \rho + \frac{1}{\nu}\sum_{i}{\xi_{i}} $$ The rest of the argument below follows the premise of the wrong loss function.




such that $$ w^{T}\cdot\Phi(x_{i}) \geq \rho - \xi_{i} $$ $$ \xi_{i} \geq 0. $$

It seems to me there there is nothing in these equations that stop variable $\rho$ from increasing without bounds in trying to minimize $L$.

Here's my argument: rewriting the inequalities, we obtain: $$ \xi_{i} - \rho \geq -w^{T}\cdot\Phi(x_{i}) $$ $$ \xi_{i} - \rho \geq -\rho $$ which are graphically represented below. The inequalities means that the offset slack $\xi_{i} - \rho$ lies above the blue line.

Graphical representation of inequalities, the offset slack lies above the blue line.

From this plot we can see that if a projected data point $w^{T}\cdot\Phi(x_{in})$ already lies inside the margin ($\leq \rho$), then increasing $\rho$ does nothing to change its offset slack $\xi_{in} - \rho$.

Conversely, if a point lies outside the margin, then increasing $\rho$ will have an effect of decreasing $\xi_{out} - \rho$.

Therefore, it seems like to minimize $L$, we are allowed to increase $\rho$ until $\rho = max(w^{T}\cdot\Phi(x_{i}))$ i.e. all the points lie on the 'wrong' side of the margin. Surely, this cannot be right. Am I missing anything in my logic?

$\endgroup$
  • 1
    $\begingroup$ If you rearrange the inqualities you have $\xi_i - \rho \geq -w^T \phi(x_i)$, note the minus sign on the right. $\endgroup$ – MotiN May 15 '18 at 11:49
  • $\begingroup$ @MotiN you are right! The mistaken has been corrected. $\endgroup$ – networker May 15 '18 at 16:47
0
$\begingroup$

The short answer is: I have been fooled by parentheses. The correct reading of the loss function is

$$ L = \frac{1}{2}||w^{2}|| + (\frac{1}{\nu}\sum_{i}{\xi_{i})} - \rho. $$

Therefore there is tension between increasing $\rho$ for the last term and the increase in the sum of slacks in the middle term as $\rho$ increases.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.