In most literature on Support Vector Machines,we start with a decision boundary $\vec w\cdot \vec x+b=0$. The support vectors are then assumed to lie on the planes $\vec w\cdot \vec x+b=1$ and $\vec w\cdot \vec x+b=-1$. How can we assume that the support vectors will always be equidistant from the decision boundary? Or is it that the algorithm starts with two random points, assume that they are the support vectors, then call the median plane of the support vectors the decision boundary.

I find it hard to visualize how the discovery of the best decision boundary in the second case can be analytical in nature.

  • $\begingroup$ It appears to me that you asked the same question twice. Can you delete one of the two questions? $\endgroup$ – Ferdi Oct 23 '18 at 11:43
  • 1
    $\begingroup$ It must have happened inadvertently. I'll delete the duplicate. $\endgroup$ – farhanhubble Oct 23 '18 at 11:43

You can start with no notion of "support vectors" -- only with the max-margin objective / constraint. Then you run an appropriate optimization procedure to optimize the objective.

The SVM constraints are $y_i(w^Tx_i+b) \geq 1$ for all $i$. After you run your solver, some of these constraints will be "tight" -- the $\geq$ will actually be an exact $=$. And it is exactly those points which lie on the edge of the margin, and are called "support vectors".

| cite | improve this answer | |
  • $\begingroup$ The expression for max margin $2 \over |w|^2$ is based on the symmetry of the gutters right? $\endgroup$ – farhanhubble Oct 23 '18 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.