4 edited tags
3 tweaked mathjax for greater readability

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $$K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$$. I: $$K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$$ I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $$w^*=\sum_{i \in SV} h_i y_i x_i$$ and $$w^*=\sum_{i \in SV} h_i y_i x_i$$ and the offset of the hyperplane $$b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$$ respectively $$b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$$ respectively, where x$$x$$ is a list of my training vectors, y$$y$$ are their respective labels ($$y_i \in \{-1,1\}$$), h$$h$$ are the Lagrangian coefficients and SV$$SV$$ is a set of support vectors. After that, I can use $$w^*$$ and $$b^*$$ alone to easily classify: $$c_x=sign(w^Tx+b)$$$$c_x=\text{sign}(w^Tx+b)$$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $$K(x,y)=\phi(x)\phi(y)$$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $$K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $$w^*=\sum_{i \in SV} h_i y_i x_i$$ and the offset of the hyperplane $$b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$$ respectively, where x is a list of my training vectors, y are their respective labels ($$y_i \in \{-1,1\}$$), h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $$w^*$$ and $$b^*$$ alone to easily classify: $$c_x=sign(w^Tx+b)$$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $$K(x,y)=\phi(x)\phi(y)$$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function: $$K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$$ I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $$w^*=\sum_{i \in SV} h_i y_i x_i$$ and the offset of the hyperplane $$b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$$ respectively, where $$x$$ is a list of my training vectors, $$y$$ are their respective labels ($$y_i \in \{-1,1\}$$), $$h$$ are the Lagrangian coefficients and $$SV$$ is a set of support vectors. After that, I can use $$w^*$$ and $$b^*$$ alone to easily classify: $$c_x=\text{sign}(w^Tx+b)$$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $$K(x,y)=\phi(x)\phi(y)$$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

2 added 21 characters in body

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $$K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $$w^*=\sum_{i \in SV} h_i y_i x_i$$ and the offset of the hyperplane $$b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$$ respectively, where x is a list of my training vectors, y are their respective labels ($$y_i \in \{-1,1\}$$), h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $$w^*$$ and $$b^*$$ alone to easily classify: $$c_x=sign(w^Tx+b)$$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $$K(x,y)=\phi(x)\phi(y)$$. That would make it easy. HoweverNevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

SVM classification with RBF

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $$K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $$w^*=\sum_{i \in SV} h_i y_i x_i$$ and the offset of the hyperplane $$b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$$ respectively, where x is a list of my training vectors, y are their respective labels, h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $$w^*$$ and $$b^*$$ alone to easily classify: $$c_x=sign(w^Tx+b)$$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $$K(x,y)=\phi(x)\phi(y)$$. That would make it easy. However, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $$K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $$w^*=\sum_{i \in SV} h_i y_i x_i$$ and the offset of the hyperplane $$b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$$ respectively, where x is a list of my training vectors, y are their respective labels ($$y_i \in \{-1,1\}$$), h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $$w^*$$ and $$b^*$$ alone to easily classify: $$c_x=sign(w^Tx+b)$$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $$K(x,y)=\phi(x)\phi(y)$$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

1