If we're interested in classifying, we only need the decision boundary. I know I'm missing something.
Is soft margin the smallest length on both sides of the decision boundary such that all misclassifications lie inside it? In that case, do we treat and classify the observation inside the soft margin differently or does it just give a measure of accuracy.
In soft margin SVM, we start with loss finction
$$ min \space\space \space \frac{1}{2}\lVert \beta \rVert ^2 + \gamma\sum_{i} \xi_i $$
$$ subject \space to \space\space\space y_i(\beta^Tx_i + \beta_0) \ge 1 - \xi_i \\\xi_i \ge 0 $$
The margin here also remains the unit distance from the decision boundary, but, we allow some points to cross this margin by introducing the term $\xi_i$, which allows the points to cross the margin (In hard margin, we strictly have the restriction $y_i(\beta^Tx_i + \beta_0) \ge 1$, but when the data is not linearly separable, we can not satisfy this constrain. The optimization gets infeasible in such cases.). We also penalize the points violating the margin by introducing $\gamma\sum_{i} \xi_i$ in objective function.
So, the margin in the soft margin is also same as one in hard margin. (i.e. the unit distance from decision boundary). I'll explain how this margin is unit distance as follows.
The distance of any point from the decision boundary is $\beta^{'T}x_i + \beta_0^{'}$. Now if, the data is linearly separable, then, intuitively, we try to fit decision boundary such that the data is separated as much as possible. In other words, we try to fit the decision boundary such that the nearest points from this boundary are at maximum possible distance(Read this line twice, if you don't get it).
So, let's say this optimal distance is $c$. Then, we will have,
$\beta^{'T}x_i + \beta_0^{'}\ge c \space \space\space$ for every point in the dataset. Dividing both sides by c, we get,
$\beta^Tx_i + \beta_0 \ge 1$. where $\beta = \frac{\beta^{'}}{c}$.
In other words, we can always obtain such unit distance boundary if data is linearly separable. This boundary is called margin, and the points on this boundary (points on the margin) are called support vectors.
Now, coming back to your question, In soft margin, the margin is this unit distance boundary. The points for which $y_i(\beta^Tx_i + \beta_0) = 1$, they are support vectors and they lie on the margin. The points for which $y_i(\beta^Tx_i + \beta_0) \ge 1 - \xi_i,\space$where $\xi_i \gt 0$, they are violators of the margin. They lie inside the margin, or, they even can cross the decision boundary.The optimal $\xi$ which controls the points that can violate the margin in soft margin is obtained by optimization of the loss function.
Coming to how classification is done, it is done based on which side of the decision boundary the point lies. The margin does not play any role in the classification of points. Its solely based on your decision boundary.
