Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
4 edited tags
| link
3 tweaked mathjax for greater readability
source | link

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$. I: $$ K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right) $$ I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $w^*=\sum_{i \in SV} h_i y_i x_i$ and $$ w^*=\sum_{i \in SV} h_i y_i x_i $$ and the offset of the hyperplane $b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$ respectively $$ b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right) $$ respectively, where x$x$ is a list of my training vectors, y$y$ are their respective labels ($y_i \in \{-1,1\}$), h$h$ are the Lagrangian coefficients and SV$SV$ is a set of support vectors. After that, I can use $w^*$ and $b^*$ alone to easily classify: $c_x=sign(w^Tx+b)$$c_x=\text{sign}(w^Tx+b)$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $K(x,y)=\phi(x)\phi(y)$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $w^*=\sum_{i \in SV} h_i y_i x_i$ and the offset of the hyperplane $b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$ respectively, where x is a list of my training vectors, y are their respective labels ($y_i \in \{-1,1\}$), h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $w^*$ and $b^*$ alone to easily classify: $c_x=sign(w^Tx+b)$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $K(x,y)=\phi(x)\phi(y)$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function: $$ K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right) $$ I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $$ w^*=\sum_{i \in SV} h_i y_i x_i $$ and the offset of the hyperplane $$ b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right) $$ respectively, where $x$ is a list of my training vectors, $y$ are their respective labels ($y_i \in \{-1,1\}$), $h$ are the Lagrangian coefficients and $SV$ is a set of support vectors. After that, I can use $w^*$ and $b^*$ alone to easily classify: $c_x=\text{sign}(w^Tx+b)$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $K(x,y)=\phi(x)\phi(y)$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

2 added 21 characters in body
source | link

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $w^*=\sum_{i \in SV} h_i y_i x_i$ and the offset of the hyperplane $b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$ respectively, where x is a list of my training vectors, y are their respective labels ($y_i \in \{-1,1\}$), h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $w^*$ and $b^*$ alone to easily classify: $c_x=sign(w^Tx+b)$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $K(x,y)=\phi(x)\phi(y)$. That would make it easy. HoweverNevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

SVM classification with RBF

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $w^*=\sum_{i \in SV} h_i y_i x_i$ and the offset of the hyperplane $b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$ respectively, where x is a list of my training vectors, y are their respective labels, h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $w^*$ and $b^*$ alone to easily classify: $c_x=sign(w^Tx+b)$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $K(x,y)=\phi(x)\phi(y)$. That would make it easy. However, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function $K(x_i,x_j)=\exp\left(-\frac{||x_i-x_j||^2}{2\sigma^2}\right)$. I know how a normal linear SVM works, that is, after solving the quadratic optimization problem (dual task), I compute the optimal dividing hyperplane as $w^*=\sum_{i \in SV} h_i y_i x_i$ and the offset of the hyperplane $b^*=\frac{1}{|SV|}\sum_{i \in SV}\left(y_i - \sum_{j=1}^N\left(h_j y_j x_j^T x_i\right)\right)$ respectively, where x is a list of my training vectors, y are their respective labels ($y_i \in \{-1,1\}$), h are the Lagrangian coefficients and SV is a set of support vectors. After that, I can use $w^*$ and $b^*$ alone to easily classify: $c_x=sign(w^Tx+b)$.

However, I don't think I can do such a thing with an RBF kernel. I found some materials suggesting that $K(x,y)=\phi(x)\phi(y)$. That would make it easy. Nevertheless, I don't think such a decomposition exists for this kernel and it's not mentioned anywhere. Is the situation so that all the support vectors are needed for the classification? If so, how do I classify in that case?

Thanks a lot for your help.

1
source | link