Linked Questions

69
votes
4answers
39k views

What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive ...
28
votes
3answers
16k views

Feature map for the Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
5
votes
1answer
5k views

How to calculate decision boundary from support vectors?

I want to obtain decision boundary of SVM using OpenCV 2.4.11, but it seems that it's not returning it explicitly, but only support vectors. How we can calculate decision boundary from support ...
3
votes
2answers
4k views

Getting distance of points from decision boundary with linear SVM?

I posted this originally in Stack Overflow but realize it might be more of a statistics question. I am using SKLearn to run SVC on my data. ...
7
votes
1answer
963 views

Are “kernel methods” and “reproducing kernel Hilbert spaces” related?

Are "kernel methods" and "reproducing kernel Hilbert spaces" related? Specifically, is the "kernel" used in the term "kernel methods" the same (type of) "kernel" as that used in the term "reproducing ...
5
votes
1answer
560 views

SVM basic theory?

I have some questions about SVM: In SVM there is a nonlinear and linear SVM. What is the difference between them? To do classification in SVM, we will find the linearly separable boundary (hyperplane)...
0
votes
1answer
501 views

Dimensionality of the Gaussian Kernel

I have read that the dimensionality of the feature map of the Gaussian Kernel is infinite. However I saw another post (Kernel SVM) stating that the feature map for a Kernel SVM maps to a ...
4
votes
1answer
135 views

Kernels in SVM primal form

For a soft margin SVM in primal form, we have a cost function that is: $$J(\mathbf{w}, b) = C {\displaystyle \sum\limits_{i=1}^{m} max\left(0, 1 - y^{(i)} (\mathbf{w}^t \cdot \mathbf{x}^{(i)} + b)\...
1
vote
0answers
215 views

SVM kernel mapping, finding boundaries in projected space

I have a question about the support vector machine (SVM) kernel trick. How do you find the boundaries of the training data set in kernel projected space? Is that the same boundaries as you can obtain ...
0
votes
2answers
100 views

Why are SVMs hard to fit?

I often hear the following complaint from people: "SVMs work really well WHEN they actually work." By "work" I mean that the algorithm will actually finish running. Are SVMs difficult to fit in ...
1
vote
0answers
144 views

SVM: (Using the RBF as a Kernel) vs (Using the RBF to create a new set of features)

I apologize for the verbose description, but (after searching several places for an answer) maybe the best way to phrase it is to lay things out explicitly. Say we are trying to build an SVM model ...