# Questions tagged [kernel-trick]

Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

605 questions
Filter by
Sorted by
Tagged with
65 views

### Does training loss go to zero in kernel regression?

High Level Problem Statement While studying kernel regression, after playing around with some linear algebra, I appear to have managed to "prove" that training loss always goes to zero. I'll ...
18 views

### What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $k$ over any space $\mathcal{X}$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $k$. For example, Radial basis ...
31 views

### Random Fourier Features vs Eigenfunctions for Gaussian Process Kernel Approximations?

Say we define kernels in Gaussian processes. There are two approaches to approximating them: random fourier features and eigenfunctions of the kernel. What are the tradeoffs to using each? If we ...
33 views

### why is rbf kernel svm a non-parametric algorithm?

I was reading up the difference between parametric and non-parametric models on this site: https://sebastianraschka.com/faq/docs/parametric_vs_nonparametric.html It says that linear SVM is parametric ...
16 views

### Dimensionality problem in dual SVM regression formulation

Consider the Boston Housing dataset. If we denote the house price with $y$ and the vector of predicting variables with $x$, then the Kernel SVMs are solved by considering the following dual convex ...
39k views

### What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive ...
13 views

### How is a polynomial kernel with infinite degree different from RBF Kernel?

I was reading about polynomial and RBF Kernels. According to my understanding: Polynomial kernels with degree >1 map the non-linear data into a higher dimensional feature space. Data that aren't ...
4 views

### Is there a relationship between the reproducing property of RKHS and eigenpair integral equations?

When we solve for the eigenpairs of a kernel we have the following equation: \begin{align} \lambda\phi(x)&=\int k(t,x)\phi(t)dt \end{align} where the right hand side can be interpreted as an inner ...
9 views

### Prove that the mixed partial derivative of a valid kernel is still a valid kernel

I have a vague memory of reading somewhere that the mixed partial derivative of a valid kernel is still a valid kernel but I cannot seem to find the original source. Does anyone have anything on it?
25 views

### What is the difference between kernel function and kernel trick?

My question is regarding the SVM topic. What is the difference between kernel function and kernel trick? Are they same and refer to the same thing?
15 views

### Are haar bases eigenfunctions for any kernel?

Are haar wavelet bases eigenfunctions for any kernel? If so, what Kernel is it, and how would we find the eigenvalues?
17k views

### Feature map for the Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
78 views

### How to understand mapping function of kernel?

For a kernel function, we have two conditions one is that it should be symmetric which is easy to understand intuitively because dot products are symmetric as well and our kernel should also follow ...
29 views

### Is it necessary that an explicit feature map exists with all kernels? [duplicate]

Consider the Radial Basis Kernel $$K(x,z) = \exp\left(-\frac{\|x−z\|^2}{2\sigma^2}\right)$$ Is it possible to find a feature map in this case? Is it necessary that an explicit feature map exists with ...
223 views

### How to Interpret output Coefficients of Linear Support Vector Regression?

I'm looking to interpret the output from my SVR model. I know that with SVM you can't directly interpret the coefficients of the model but that you first have to take a dot product With that said, ...
29 views

### Compare distributions using Maximum Mean Discrepancy (MMD)

I use MMD distance to run a permutation test and decide whether two sample distributions come from the same distribution or not. For the MMD, I use a gaussian kernel, the bandwidth of which I select ...
196 views

### Better classification performance when using an RBF kernel function in high dimensional space?

I'm learning about SVM's and understand that boosting something into a higher dimension can sometimes help separate the data better. However, if I were to perform 1 nearest neighbor with the RBF ...
37 views

### Kernel approximation with Nystroem method and usage in scikit-learn

I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn. As far as I understood, the full Gram Matrix ...
28 views

### Pros and cons of different MKL algorithms

I have been using multiple kernel learning (MKL) to train a classifier and got some exposure to the field. However, I am quite new to machine learning and I have only an intuitive understanding of the ...
16 views

### Can I customize the kernel function?

I want to know whether I can customize the kernel function? For example, the polynomial kernel is defined as: $$K(x,y) = (x^Ty+c)^d$$ Could I modify it to the following: $$K(x,y) = (||x-y||_2)^d$$ ...