1
$\begingroup$

I try to get the basic understanding behing SVM algorithm, however I have a problem with basic mathematics.

I follow the lecture Support Vector Machine.

Suppose the two classes can be separated by a hyperplane:$(w \cdot x) + b = 0$

Acoording to wikipedia, hyperplane is defined as $n(r-r_0)=0$, does it mean that $b=-w \cdot r_0$?

I tried to consider 2-dimensional case, when $w \cdot x +b =0$ is a line, but it's completely doesn't make sense, $b=-w \cdot x$, where $b$ should be a constant, how can I generalize it to a 2 dimensional case.

In addition, why $w$ is actually orthogonal to the plus and minus plane?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
2
$\begingroup$

suppose a plane (you can simplify it as a line too) w'*x + b = 0. If you take two vectors on the plane (it will be two points on the line) x1, and x2, since 

w'*x1 + b = 0
w'*x2 + b = 0

we get,

w'*(x1 - x2) = 0

which is the same form as wiki definition you provided. This vector production is actually the dot product, and equivalent to the projection of vector x1 - x2 on w. The projection is zero, indicating w is orthogonal to the plane.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you very much for your answer, you say I can simplify it as a line too, $w'*x+b=0$ but it's a "strange" line, as a line I expect to see something like y = k*x+b, but here $y=0$, so it is just X-axis? $\endgroup$
    – user16168
    Dec 13, 2013 at 7:38
  • $\begingroup$ Yes it is y=w'x+b, or f(x,b)=w'x+b. We just define f(x,b)=0 as the plane that will have the largest margin. f(x,b)=1 means the plane/line moves down by a unit distance but the slope keeps the same as w'. You can easily catch this by drawing a line y=x+1 and y=x. Here if we define x+1=0 as the control plane, x+1=1 refers to y=x, but it doesn't mean x is zero (that is not an equation) $\endgroup$
    – lennon310
    Dec 13, 2013 at 15:16
0
$\begingroup$

take your hand as the plane, and a finger as the normal direction, then it should be clear that the plane is defined by two quantities, the direction of the normal and distance - how far your finger is planted in the plane. that distance is what b is representing.

so yes b=-n.r_0

b is the distance of the hyperplane to the origin (assuming n is unit length). So all points on the plane have the same "perpendicular distance" to the origin, and that's why you have a constant b , on the one side, and a seemingly varying quantity w.x on the other side.

draw a plane, and a line joining the origin to any arbitrary point in the plane, together with the shortest line joining the origin to the plane (which is precisely the one that enters the plane at right angles). Then you can decompose all points on the plane as a vector from the origin to closest point on plane together with an orthogonal vector within the plane. So that distance from origin to closest point doesnt change and that is what is giving you b.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.