1
$\begingroup$

I've been reading a lot about SVMs and have some questions about performing classification from the SVM model produced from a package like libSVM.

From my understanding, for a linear SVM without the use of any kernel, we can apply the SVM model to perform binary classification on a test instance $\vec{x}$ by checking to see on which side of a hyperplane the instance lies on, i.e. to see if:

$f(\vec{x}) = \vec{w} \cdot \vec{x} + b$

is greater than or less than 0. It seems to me that we can precompute $\vec{w}$ as:

$\vec{w} = \sum_{i=1}^{N} \lambda_{i} y_{i} \vec{x}_i$

before we begin classification (where the lagrange multipliers $\lambda_i$ are produced from an SVM training package like libSVM).

However, if we use a (non-linear) kernel function for non-linear SVM, then am I correct in understanding that we cannot precompute anything? That is, to perform classification, we have to compute $ K(\vec{x}_i, \vec{x})$ as part of:

$f(\vec{x}) = \sum_{i=1}^{N} \lambda_{i} y_{i} K(\vec{x}_i, \vec{x}) + b$

Since $ K(\vec{x}_i, \vec{x})$ may be something fancy like a polynomial kernel $K(\vec{x}_i, \vec{x}) = (\vec{x}_i \cdot \vec{x})^{d}$, there is no "$\vec{w}$" that can precomputed.

Is this correct?

$\endgroup$

1 Answer 1

1
$\begingroup$

Basically, because you are projecting the data into a new space (higher dimensional space) or in other words, changing how you define the correlation between two points (based on the kernel function), the classifier is being calculated in that new space. You might be able to find some kind of weights in the new space, but unfortunately not for the old one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.