# Questions tagged [kernel-trick]

Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

605 questions
Filter by
Sorted by
Tagged with
223 views

### How to Interpret output Coefficients of Linear Support Vector Regression?

I'm looking to interpret the output from my SVR model. I know that with SVM you can't directly interpret the coefficients of the model but that you first have to take a dot product With that said, ...
283 views

### Are there examples of covariance functions used in Gaussian processes with negative non-diagonal elements?

I've been searching through numerous kernels used in Gaussian processes, and one common feature is that the covariance matrices always have only positive elements. Yet the only requirement on the ...
197 views

### Maximum Mean Discrepancy Implementation

I am just beginning to learn about MMD as a way to measure the difference between two probability distributions using this tutorial. I want to implement it code-wise but I don't understand it ...
251 views

### Why isn't a gaussian kernel subject to the curse of dimensionality?

This has been bugging me for a while now. I understand from this answer why gaussian kernels are effective. But I can't wrap my head around the intuition of why the infinite dimensional feature map 𝜙(...
107 views

### Is kernalized linear regression parametric or nonparametric?

We know that for linear regression, we can predict: $$\hat{y} = w^Tx +b$$ Where $w$ is the parameter that minimizes the square loss. It is easy to prove that for the final solution using gradient ...
30 views

### A derivation regarding kernel regression for the support vector machine

THis is from the Elements of Statistical Learning book page 437 in the section of support vector machine. Can anyone give me some hint for the missing derivation steps for why 12.49 is true (as seen ...
58 views

### Expansion of inner product for polynomial kernel for SVMs

On page 424 in "The Elements of Statistical Learning" by Hastie et al (2013) (https://web.stanford.edu/~hastie/Papers/ESLII.pdf), we see the following expansion of a polynomial kernel with degree 2: ...
29 views

### Relevant Dimension Estimation: Showing that $\sum_{i=1}^N s_i^2 = ||y||^2$

In relevant dimension estimation, we are given a Kernel Matrix $K \in \mathbb{R}^{n \times n}$, where $K_{ij} = k(x_i, x_j)$. We then compute the kernel eigenvector from the multiple solutions of the ...
365 views

### Can someone provide a brief explanation as to why reproducing kernel Hilbert space is so popular in machine learning?

I thought functional analysis was long thought to be old fashioned and generally a dead research area. It seems that all of a sudden there is a huge fascination with so-called reproducing kernel ...
35 views

180 views

### How to choose hyper-parameter for Gaussian Process kernels?

I'm trying to fit Gaussian Process in scikit-learn, and start with using kernel = RBF_1 + RBF_2 + whitekernel(sum of two RBF kernels with different length_scale and ...
646 views

### Why do we need the gamma parameter in the polynomial kernel of SVMs?

The polynomial kernel is sometimes defined as just: $$K(x,y):=(\left<x,y\right>+c)^d$$ with two parameters: the degree $d$ and constant coefficient $c$. But others (e.g., libsvm, and sklearn ...
189 views

### Precomputed Kernels for Support Vector Machines (SVM)

To calculate the linear kernel matrix for some training matrix X with dimensions n x d where d is the number of features and n is the number of data points, we can simply do: $X * X^T$. The result is ...
972 views

### Finding the feature map corresponding to a specific Kernel? (Polynomial Kernels)

I am just getting into machine learning and I am kind of confused about how to show the corresponding feature map for a kernel. For example, how would I show the following feature map for this kernel? ...