# Tag Info

## Hot answers tagged kernel-trick

### How to intuitively explain what a kernel is?

Kernel is a way of computing the dot product of two vectors $\mathbf x$ and $\mathbf y$ in some (possibly very high dimensional) feature space, which is why kernel functions are sometimes called "...
• 8,407

### How to intuitively explain what a kernel is?

A visual example to help intuition Consider the following dataset where the yellow and blue points are clearly not linearly separable in two dimensions. If we could find a higher dimensional space ...

### What makes the Gaussian kernel so magical for PCA, and also in general?

I think the key to the magic is smoothness. My long answer which follows is simply to explain about this smoothness. It may or may not be an answer you expect. Short answer: Given a positive ...
• 2,043

### How to intuitively explain what a kernel is?

A very simple and intuitive way of thinking about kernels (at least for SVMs) is a similarity function. Given two objects, the kernel outputs some similarity score. The objects can be anything ...
• 2,375
Accepted

### Difference between Primal, Dual and Kernel Ridge Regression

Short answer: no difference between Primal and Dual - it's only about the way of arriving to the solution. Kernel ridge regression is essentially the same as usual ridge regression, but uses the ...
• 8,407
Accepted

### What are the advantages of kernel PCA over standard PCA?

PCA (as a dimensionality reduction technique) tries to find a low-dimensional linear subspace that the data are confined to. But it might be that the data are confined to low-dimensional nonlinear ...
• 96.4k

### How can SVM 'find' an infinite feature space where linear separation is always possible?

This answer explains the following: Why perfect separation is always possible with distinct points and a Gaussian kernel (of sufficiently small bandwidth) How this separation may be interpreted as ...
• 10.2k
Accepted

### Is there any supervised-learning problem that (deep) neural networks obviously couldn't outperform any other methods?

Here is one theoretical and two practical reasons why someone might rationally prefer a non-DNN approach. The No Free Lunch Theorem from Wolpert and Macready says We have dubbed the associated ...
• 19.6k

### How to prove that the radial basis function is a kernel?

I'll add a third method, just for variety: building up the kernel from a sequence of general steps known to create pd kernels. Let $\mathcal X$ denote the domain of the kernels below and $\varphi$ the ...
• 22.6k
Accepted

### What is the rationale of the Matérn covariance function?

In addition to @Dahn's nice answer, I thought I would try to say a little bit more about where the Bessel and Gamma functions come from. One starting point for arriving at the covariance function is ...
• 2,746
Accepted

### Nystroem Method for Kernel Approximation

Let's derive the Nyström approximation in a way that should make the answers to your questions clearer. The key assumption in Nyström is that the kernel function is of rank $m$. (Really we assume ...
• 22.6k

### Is there any supervised-learning problem that (deep) neural networks obviously couldn't outperform any other methods?

Somewhere on this playlist of lectures by Geoff Hinton (from his Coursera course on neural networks), there's a segment where he talks about two classes of problems: Problems where noise is the key ...
• 4,737

### What is the rationale of the Matérn covariance function?

I do not know, but I found this question very interesting and here's what I got after a bit of reading on it. For certain values of $\nu$, the Matérn covariance function can be expressed as a product ...
• 518

### What makes the Gaussian kernel so magical for PCA, and also in general?

I will do my best to answer this question not because I'm an expert on the topic (quite the opposite), but because I'm curious about the field and the topic, combined with an idea that it could be a ...
• 8,005

• 5,112
Accepted

### Proof that $K(x,y) = f(x)f(y)$ is a kernel

$\sum_{i=1}^n\sum_{j=1}^nK(x_i, x_j)c_ic_j=\sum_{i=1}^n\sum_{j=1}^nf(x_i)f(x_j)c_ic_j = (\sum_{i=1}^nf(x_i)c_i)^2 \geq 0$
• 5,616