SVM decision boundary conditions : derivation problem I was trying to understand the derivation of SVM decision boundary.
Suppose my decision boundary is y-x-1=0.
Now in the book it was written that (y-x-1) = +/-(1), are the equation of the maximum margins. 
But my question is why are we choosing a distance of 1 from the decision boundary ? How do we even know that we would be able to hit the support vectors just at the distance of +/-(1) from the decision boundary ? 
Moreover how can we say, that if we are able to hit a support vector at a distance of +1 from decision boundary then it will be there exactly at -1 (and not at -3, -5, -10... etc for that matter) ? i.e. how are we so sure that at exactly +/-(1) "on both sides" from decision boundary we will be able to hit support vectors.
 A: You seem to misunderstand support vectors. Support vectors are not at distance exactly 1 to the separating hyperplane.
Support vectors arise as a result from the constraints in the optimization problem. The constraints basically require that every data instance should be out of the classification margin and on the right side of the separating hyperplane defined by $\mathbf{w}$ and $b$. The following Wikipedia image illustrates this idea nicely:

For positive instances $\mathbf{x}_p$, we require that $\mathbf{w}^T\mathbf{x}_p - b \geq 1$ and for negative instances $\mathbf{x}_n$ this means $\mathbf{w}^T\mathbf{x}_n - b \leq -1$. Through clever use of the binary class labels (e.g. $y_i \in \{-1, +1\}$) we can write this in a single form. For notational simplicity, these constraints can be written as follows for linear SVM:
$$y_i\big(\mathbf{w}^T\mathbf{x}_i - b\big) \geq 1 - \xi_i, \quad \forall i$$
where $\mathbf{y}$ is the vector of training labels, $\mathbf{w}$ is the separating hyperplane, $\phi(\cdot)$ is the embedding function and $b$ is the bias. $\xi$ is a vector of slack variables which allow soft margin classification. These slack variables are positive when the constraint is violated and zero otherwise. Such a constraint exists for every training instance. 
Support vectors are those instances for which $\xi_i$ is necessarily nonzero, e.g. $\mathcal{S} = \{i : y_i\big(\mathbf{w}^T\mathbf{x}_i - b\big) < 1\}$. This occurs for instances that are either misclassified or those close to the separating hyperplane (distance less than 1). For example: if there were positive instances (black dots) below the dashed line $\mathbf{w}^T\mathbf{x}-b = 1$ on the figure, those instances would also be support vectors.
