3
$\begingroup$

I am using the SVMStruct function in MATLAB (with RBF kernel) to classify my data, and it works great. But now I need to compare the distance from the data points to the hyperplane, or to find the data point that is closest to the hyperplane. I don't find a function in MATLAB to do that, or even how this can be done. Could someone please suggest?

$\endgroup$
3
$\begingroup$

You can get the hyperplane only in the case of linear kernel (a.k.a dot-product) case. Here, the input for the computation are (based on what I could interpret from the documentation and a helpful thread)

  1. SVMStruct.Bias (call it $b$)
  2. SVMStruct.SupportVectors (call it $\{x_j\}$) (Note: These are data points closest to the hyperplane)
  3. SVMStruct.Alpha (call it $\{\alpha_j\}$)

The output is: $w^T = [(\sum_{j}\alpha_jx_j)^T\;\; b]$. The distance of every training point to the hyperplane specified by this vector $w$ is $w^T[x_i]/||w||_2$.

For RBF kernel, the representation of the classifier or regressor is of the form $\sum_{i=1}^n \alpha_i K(x_i,x)$ where $n$ is the number of training examples and $K$ is the kernel we choose and $\{x_i\}$ are our training data points. The hyperplane lives in a possibly higher (even infinite) dimension. This hyperplane is of course different from the decision boundary (which is non-linear) which you may visualize when you have only 2-dimensional features.

Notation: vectors are in column format.

$\endgroup$
  • $\begingroup$ Thanks, @Theja it really helps. The thread you gave is also very helpful. I just got the question, in the equation $w^T = [(\sum_{j}\alpha_jx_j)^T\;\; b]$ , is it supposed to be $w^T = [(\sum_{j}\alpha_jx_j)^T+ b\;]$ ? $\endgroup$ – AshX Oct 4 '13 at 15:58
  • $\begingroup$ $w$ is a vector with its first d coordinates being $\sum_j\alpha_j x_j$ and the d+1 coordinate being $b$. Here, d is the dimension of the feature vector. $\endgroup$ – Theja Oct 5 '13 at 16:44
5
$\begingroup$

What about just computing it explicitly? If a hyperplane is defined as $\langle \vec a, \vec x \rangle =0$, than the distance $$ d(\vec x_0) = \frac{\langle \vec a, \vec x_0 \rangle}{\| \vec a \|} $$ Programming it in matlab is easy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.