# Questions tagged [kernel-trick]

Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

743 questions
Filter by
Sorted by
Tagged with
6 views

### SVM Kernels feature map and feature space [closed]

Can someone help me to find the feature space of kernel K(x,y) = exp(-|x-y|^2)
22 views

### Non-stationary Random Fourier Features

Random Fourier Features (RFFs) were introduced by A. Rahimi and B. Recht in their 2007 publication Random Features for Large-Scale Kernel Machines. RFFs are based on Bochner's theorem, which applies ...
20 views

### Identifiability of models on RKHS

I have just started learning about using reproducing kernel hilbert spaces for regularisation in machine learning. I am looking for some examples of reproducing kernels that produce identifiable and ...
• 41
46 views

### Feature maps of the chi-squared kernel

The additive chi-squared kernel for histograms is defined as $$K(x,y)= \sum_{i=1}^n \frac{2x_i y_i}{x_i + y_i}$$ Is this kernel positive definite on histograms? And if so, is there a known expression ...
17 views

### Method of evaluating the feature map of a polynomial kernel feature mapping

I'm attempting to implement an adaptive kernel Kalman filter following this paper https://arxiv.org/abs/2203.08300, but I'm struggling to find a method of evaluating the feature mapping for a ...
62 views

### Is the transformation implied by a positive-type kernel well-defined?

I’ve been trying to get my head around the particularity of the Hilbert space that a positive-type (equiv. positive definite) kernel represents an inner product on, and was hoping for some help in ...
18 views

### In Gaussian Process Regression, what kinds of information can you *not* put in the kernel as opposed to the mean?

For example, suppose you want to learn some structure for the mean and then you also have some kernel. Is is sometimes not possible to put most things in the kernel? For example, consider ...
• 269
15 views

### Optimal kernel regression with random target functions

The classic kernel regression problem fixes a target function $f^*$ that we seek to learn, and says that for a dataset/observations $D = \{x_i, y_i\}_{i=1}^n$ where $y(x) = f^*(x) + \epsilon$, for ...
15 views

• 1,813
26 views

### Is kernelization just a bases transform?

I am trying to understand the concept of kernelization, however, it appears to me that explanations from the machine learning community are rather confusing. For example, in the Bishop book (eq. 6.3-6....
• 217
1 vote
40 views

### Are low-rank kernel approximations implementing implicit regularization?

Consider a kernel estimation problem as follows. We have functions $f^* \sim GP(0, C^*)$ drawn from a Gaussian process. We want to construct a kernel $K$ that does well in regressing functions drawn ...
1 vote
26 views

### Mathematical steps for solving the Kernel trick for Support Vector Machines (SVM)s

I am trying to understand the math behind the Support Vector Machines (SVM) Kernel Trick but there isn't any real source online for the mathematical steps to I can follow to see how its solved. One ...
• 11
6 views

### How are kernel similarities viable replacements in SVMs?

Support vector machine (SVM) is a supervised learning algorithm. It draws hyperplanes to separate data points of different classes. The objective function involves inner products of pairs of feature ...
• 249
61 views

### Is there a connection between the Kernels in Statistics and Linear Algebra?

According to this question, the etymology of the terms is related, and Kernel is used to mean the "core" of something. In general it seems to refer to an unchanging transformation at the ...
• 279
30 views

• 217
26 views

### Why we need RKHS? [duplicate]

I'm pretty informed of the concept of RKHS and relative facts like Hilbert spaces or kernel function. But why we relate kernel function to linear algebra and raise specificly the concept of RKHS? Is ...
1 vote
26 views

### Would removing covariates correlated with model residuals (errors) from training improve prediction in machine-learning or deep-learning? [closed]

Would removing covariates correlated with model residuals (on training data) improve prediction in machine-learning or deep-learning or make the weights calculated more interpretable or useable to ... 35 views

### Is it possible to apply the kernel trick to a "mahalnobis distance learner" such as GLS?

1.https://arxiv.org/pdf/0804.1441.pdf 2.https://www.sciencedirect.com/science/article/abs/pii/S0925231210001165 These papers describe kernelizing a mahalanobis distance learner. I am interested in ... 1 vote
39 views

### Is there a multilinear kernel principal components analysis?

PCA can be extended to kPCA using the kernel trick. MPCA is a multilinear extension of PCA that involves multiple matrices for the different modes of the data tensor. Can MCPA be similarly extended ...
• 6,135
1 vote
40 views

### Is there a difference between kernel PCA with a non-linear kernel vs PCA with a non-linear change of variables?

I see that kernel PCA with a linear kernel is the same as PCA. On Wikipedia's introduction of the kernel to PCA they suggest that there exists a non-trivial arbitrary choice of map $\Phi$ that is ...
• 6,135
13 views

### Objective of maximal margin classifier with quadratic decision boundary

What is the objective for minimization for a maximal margin classifier with quadratic decision boundary of the form $x^{T}Ax + b^Tx + c =0$?
• 273
81 views

### Relation between generalization bounds of Kernel Ridge Regression and largest eigenvalue of the kernel Gram matrix

Consider a positive-definite, symmetric function $k(x_1, x_2)$ which is used, given the dataset $\{(x_i, y_i)\}_{i=1}^m$, to construct the Gram matrix $K = [k(x_i, x_j)]_{i,j \in 1, ..., m}$. What is ...
• 41
1 vote
Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $\mathbf{K}$ and standard PCA via SVD of the input $\mathbf{X}$. In standard PCA as far as I know we can derive \$\mathbf{S}...