0
$\begingroup$

I don't have a background of computer science, but i have my major in mathematics. I am interested in Support Vector Machines. I went through the theory as well as some practical examples. My problem is that I understood all the concepts, like separating hyperplanes, kernels and the importance of the parameters C and gamma.

But what I would like to know is, if we implement this algorithm of Support Vector Machines in software, then what outputs the software will give us and how do we implement those outputs?. What are the important things to note down?

$\endgroup$

2 Answers 2

2
$\begingroup$

The output of an SVM solver are the coefficients in the resulting model. The decision function in an SVM for a test instance $\mathbf{z}$ is as follows: $$ f(\mathbf{z}) = \mathbf{w}^T\phi(\mathbf{z}) + b, \\ $$ where $\mathbf{w}$ is the separating hyperplane, $\phi(\cdot)$ is the embedding function and $b$ is the bias term. A binary label is assigned based on the sign of $f(\mathbf{z})$. Note that this formulation contains the inner product between the separating hyperplane $\mathbf{w}$ in feature space and the feature-space representation of the test instance $\mathbf{z}$. For some kernels, the embedding function is unknown or leads to an infinite dimensional vector (such as the Gaussian kernel).

Due to the representer theorem and the kernel trick, this can be rewritten as follows: $$ f(\mathbf{z}) = \sum_{i\in \mathcal{S}} \alpha_i y_i \kappa(\mathbf{x_i}, \mathbf{z}) + b, $$ with $\alpha$ the vector of support values, $\mathbf{y}$ the vector of training labels, $\mathcal{S}$ the set of support vectors and $\kappa(\mathbf{x}_i,\mathbf{x}_j) = \phi(\mathbf{x}_i)^T\phi(\mathbf{x}_j)$ the kernel function.

An SVM model is entirely defined by the coefficients $\alpha$ and $b$ as solution of the training problem and the support vectors with their associated labels.

$\endgroup$
1
$\begingroup$

Adding on the previous answer, when solving the problem, you get the values for $\alpha_{i}$, that is, you know which samples are the support vectors, and the offset $b$.

But also you are talking about implementing SVMs. Generally, you do not want to ask for those parameters, as a user, but:

  • for a given sample, what label does the SVM assign to it?

  • what are the support vectors?

  • the value of the decision function. This gives you the margin plus the sign (depending on what side of the hyperplane the sample lies), and can be use to generate confidence values for the predictions of the SVM.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.