0
$\begingroup$

I would like to know if the linear-SVM-without-offset solver: $$\min \frac{1}{2}\|w\|^2+C\sum_{i=1}^m \xi_i, \quad \mbox{s.t.}\quad y_iw^\top x_i \geq 1-\xi_i, \quad \xi_i\geq 0 \quad \forall i=1,\ldots,m.$$ can be applied to classify linearly separable data, where the hyperplane does not pass through the origin.

Maybe a change of coordinate system would help?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

A linear SVM can't do anything a linear model cannot do, so without a bias term, the hyperplane must pass through the origin. This can be addressed by adding an extra input feature with a constant value (which is how a bias term is often implemented in linear regression).

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi @Dikran. Thanks for the answer. Could you please be more specific about "adding an extra input feature..."? My thinking: Suppose that we have already an "SVMs-without-bias" solver $F(X,y,C)$ (that returns $w$ and $\xi$). Now for an instance $(X,y,C)$, where the data $(X,y)$ are linearly separable by a hyperplane that does not pass through the origin, if one applies a suitable rotation to the data to obtain $X',y'$ (i.e. we go to another space, the same idea as "feature space" in kernels), then the above solver can be used to classify the new (rotated) features: $F(X',y',C)$. $\endgroup$
    – f10w
    Apr 3, 2014 at 18:15
  • $\begingroup$ I understand now. We can either use a rotation or a translation. What you have suggested is a translation, which is much simpler than a rotation. In this case, we have to solve another problem than "SVM-without-offset", haven't we? (I mean we cannot use the "SVMs-without-bias" solver). $\endgroup$
    – f10w
    Apr 3, 2014 at 18:19
  • $\begingroup$ If you just add an extra column of ones to $X$, then that will give an implicit bias parameter, even though you can still use the "SVMs-without-bias" solver. The extra element of $w$ will be the bias term. $\endgroup$
    – Dikran Marsupial
    Apr 4, 2014 at 8:12
  • $\begingroup$ This is exactly what I was looking forward. I read it once somewhere but then couldn't find it again. Thank you so much, @Dikran! (P/s: It would be nice if you edit the answer and replace it by this comment). $\endgroup$
    – f10w
    Apr 4, 2014 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.