Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm trying to figure out support vector machines using this resource. On page 2 it is stated that for linearly separable data the SVM problem is to select a hyperplane such that $\vec{x}_i\vec{w} + b \geq 1$ for $y_i \in 1$ and $\vec{x}_i\vec{w} + b \leq -1$ for $y_i \in -1$. I'm having trouble to understand where the right-hand side of the constraints come from?

P.S The next question would be how to show that the SVM's margin is equal to $\frac{1}{||\vec{w}||}$.

share|improve this question
2  
See if you find this useful - stats.stackexchange.com/questions/43727/… –  TenaliRaman Jan 2 '13 at 22:58
1  
yes, the answer for this question is part of the answer given here stats.stackexchange.com/a/43811/16837 –  entropy Jan 3 '13 at 1:05
add comment

2 Answers 2

up vote 3 down vote accepted

Essentially these two constraints basically require the training data to be correctly classified, and at least a certain distance from the decision threshold 0. The hyperplane that fulfils these constraints with the smallest norm of the weights will have the maximal margin. The value $\pm 1$ is essentially arbitrary, you could replace it with $\pm$ any value you like and it would merely rescale the coefficients of the hyper-plane, but without changing the decision boundary. A value of 1 is used just to keep the maths neat.

share|improve this answer
add comment

wx + b = -1 and wx + b = 1; These equations represent two parallel hyperplanes that are formed based on samples class (-1, +1). These two hyperplanes are used to optimize the distance between classes and to get optimal hyperplane. For optimization problem Lagrange multipliers are used.

enter image description here

You can find brief description on resources listed below.


Online course on Machine Learning by Andrew Ng is a great place to understand SVM and other ML algorithms: Machine Learning - Andrew Ng Hyperplane is thoroughly explained.

In order to better understand math behind the SVM, learning Optimization is the right choice. There is a great free ebook by S.Boyd: Optimization - Boyd

Here you can find SVM papers: svms.org/tutorials/

share|improve this answer

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.