What do the variables mean in the SVM objective function? For a linear SVM, the documentation tells me the formula is:
$$\frac{1}{2}w^Tw + C\sum_{i=1}^l\xi_i.$$
Please explain to me in layman's terms what $w$ and $ξ$ represent.
Is w the distance to the hyperplane, the weight of different variables, ... ?
Also, is there a difference with the formula
$$f(x)=w^T x+b$$
or is it just a different notation?
 A: Those two formulae are different things:


*

*$\frac{1}{2} w^T w + C \sum \xi_i$ is one form of the objective function, the function which is minimized over $w$, $b$, and $\xi_i$ (subject to certain constraints, which are where $b$ comes in) to find the best SVM solution.

*Once you've found the model (defined by $w$ and $b$), predictions on new data $x$ are done by finding their distance from the decision hyperplane, $f(x) = w^T x + b$.
$w$ and $b$ define the decision hyperplane, which separates positives from negatives, $\{ x \mid w^T x + b = 0 \}$. So $w$ is perpindicular to that hyperplane. $|w_j|$ is also the weight of the corresponding feature dimension: if $w_j = 0$, that feature is ignored, and if $|w_j|$ is high, that feature is important to the SVM's decision (assuming all the features are scaled similarly).
SVMs are trained by maximizing the margin, which is the amount of space between the decision boundary and the nearest example. If your problem isn't linearly separable, though, there is no perfect decision boundary and so there's no "hard-margin" SVM solution. This is why the "soft-margin" SVM was introduced, which allows some points to be on the wrong side of the margin.
$\xi_i$ is the slack variable defining how much on the wrong side the $i$th training example is. If $\xi_i = 0$, the point was classified correctly and by enough of a margin; if it's between 0 and 1, the point was classified correctly but by less of a margin than the SVM wanted; if it's more than 1, the point was classified incorrectly. ($\xi_i$ isn't allowed to be negative.) Points with $\xi_i > 0$, as well as those with $\xi_i = 0$ closest to the decision boundary, are known as support vectors because they "support" the margin. These are important in a kernel SVM because they're the only ones you need to worry about when predicting on new data.
$C$ is a parameter of the problem that defines how soft the margin should be. As $C \to \infty$, you get a hard SVM; if $C = 0$, the SVM doesn't care about getting the right answer at all and will just choose $w = 0$. In practice, you usually try a few different values of $C$ and see how they perform.
This picture (source) illustrates the different variables, though it's for a kernel SVM; just say $\phi(x) = x$ and it'll be a linear SVM.

