# Questions tagged [kernel-trick]

Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

605 questions
Filter by
Sorted by
Tagged with
27 views

### Kernel ridge regression with matrix-vector data set $S := \{ X_i, y_i \}_{i=1}^{N}$? [duplicate]

Background: For Kernel ridge regression, I have normally come across the data-set given in vector and scalar form, i.e., $\overline{S}:= \{x_i, \overline{y}_i \}$, where $x_i \in M_{n,1}(\mathbb{R})$ ...
87 views

### Kernel function for use in Kernel-PCA given a known piecewise linear true data generating process

If I know that a multivariate dataset has a piecewise-linear data generating process with known knots (or breakpoints), then what is the appropriate kernel function to use in Kernel-PCA? For example, ...
80 views

### How to combine multiple kernels of large sample datasets?

I have multiple large sample datasets in matrix format (each has 15000 rows and 5-50 columns) corresponding to different experiments. Each matrix contains the same number of samples(rows) but the ...
64 views

### Kernel functions with vector output

Kernel functions are used commonly with SVMs to make classification of non linearly separable data possible - i.e. the Kernel function provides the linear separability. But from looking at Kernel ...
3k views

### How to use the squared exponential kernel with multidimensional vector inputs?

I'm constructing an optimization (Bayesian optimization) algorithm using Java code. I have created the program, but the similarity values between inputted vectors in the kernel equation does not ...
811 views

### Fourier transform of a Gaussian process

I would like to discuss and ask a question regarding the Fourier transform of a Gaussian process, if it makes sense. For that purpose, let me describe the following situation. Let $z(s)$ be a ...
131 views

### Understanding kernel PCA when the target space is infinite-dimensional

The PCA optimization problem is known as $$\max_{U \in \mathbb{R}^{d\times r}, U^TU = I} tr(U^T\Sigma U),$$ where $\Sigma$ is a covariance matrix of a dataset $\{x_1,\dots,x_n\} \subset \mathbb{R}^d$...
149 views

### Are null space of matrix and kernel function same?

I have recently started learning about machine learning and have come across kernels and null spaces. I understand that null space is the set of all vectors that satisfy the equation A.v = 0 (Where A ...
24 views

### kernel publications

What are some contemporary papers that provide the reader with a complex overview of kernel functions used nowadays in machine learning?
37 views

### eigenvalue perturbation theory for kernel function

Let $S=\{x_i\}_{i=1}^n$ be a set of training examples, and let $K\in \mathbb{S}^n_+$ be the kernel matrix induced by $S$ and some kernel function $k$ (i.e., $K_{ij}=k(x_i,x_j)$). I was wondering how ...
371 views

### How to calculate RKHS norm of a function under given kernel transformation

This was a question asked before in mathoverflow but not yet got answered. I have the same problem when reading Srinivas et al (2010) [appendix B]'s paper. Here are my problems: Definitions: ...
211 views

24 views

### Does there exist a vector α st the equality holds?

A training set $(x_1,y_1),...,(x_m,y_m)$ is generic iff $x_i=x_j$ then $y_i=y_j$ and let's consider the following kernel $K_a(x,t)=\prod_{i=1}^n(1+(x_it_i)+(1-x_i)(1-t_i))$ Given a generic training ...
55 views

### The trace term in 2 Wassersteins metric for Gaussians

I was looking at the formula for 2 Wassersteins distance for Gaussian distribution on Wikipedia. https://en.wikipedia.org/wiki/Wasserstein_metric#Normal_distributions It satisfies all properties of a ...
197 views

### addition of a kernel and a function

I have this problem. My approach would be to take a Kernel matrix of $\ k(x,y)$ as it is positive semidefinitive. And if I am able to make $\ k'(x,y)$ a kernel matrix, then I can prove that it ...
554 views

### How to understand effect of RBF kernel for kernel PCA

I understand the math in kernel PCA and with RBF kernel, and I also understand that the RBF kernel map the data into a infinite dimensional space. I know that for SVM, mapping the data into a higher ...
57 views

137 views

139 views

1k views

### Proof that $K(x,y) = f(x)f(y)$ is a kernel

I cannot find a proof that $K(x,y)=f(x)f(y)$ is a Kernel for any real valued function $f$ and $x,y\in \mathbb{R}^n$. Obviously it is symmetric but can someone prove that it is positive semidefinite? ...
3k views

### Proof that exponential of a kernel is a kernel

How can I prove that the exponential $\exp(K)$ of a kernel function $K$ is again a kernel? I think it can be proved using Taylor expansion but I am not sure how.
1k views

### Sample from Gaussian Process across 2D

This seems like a straightforward questions; my apologies if it is already answered (I have looked). Problem: I would like to sample from a Gaussian Process (GP) prior over X and Y coordinates (e.g. ...
576 views

### Calculation of nu and gamma in one-class SVM with rbf kernel

I am using python sklearn's one-class svm classifier for anomaly detection. I would like to know can I accurately calculate the required value for nu and gamma for rbf kernel. Is there any equation or ...
I am trying to find an intuition on why we require that kernels are positive semi definite and I have found this: We are given a dataset $X$ of size $n \times d$ where $n$ is the number of samples ...