Questions tagged [kernel-trick]

Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

605 questions
Filter by
Sorted by
Tagged with
27 views

Kernel ridge regression with matrix-vector data set $S := \{ X_i, y_i \}_{i=1}^{N}$? [duplicate]

Background: For Kernel ridge regression, I have normally come across the data-set given in vector and scalar form, i.e., $\overline{S}:= \{x_i, \overline{y}_i \}$, where $x_i \in M_{n,1}(\mathbb{R})$ ...
87 views

Kernel function for use in Kernel-PCA given a known piecewise linear true data generating process

If I know that a multivariate dataset has a piecewise-linear data generating process with known knots (or breakpoints), then what is the appropriate kernel function to use in Kernel-PCA? For example, ...
80 views

How to combine multiple kernels of large sample datasets?

I have multiple large sample datasets in matrix format (each has 15000 rows and 5-50 columns) corresponding to different experiments. Each matrix contains the same number of samples(rows) but the ...
64 views

Kernel functions with vector output

Kernel functions are used commonly with SVMs to make classification of non linearly separable data possible - i.e. the Kernel function provides the linear separability. But from looking at Kernel ...
3k views

How to use the squared exponential kernel with multidimensional vector inputs?

I'm constructing an optimization (Bayesian optimization) algorithm using Java code. I have created the program, but the similarity values between inputted vectors in the kernel equation does not ...
811 views

Fourier transform of a Gaussian process

I would like to discuss and ask a question regarding the Fourier transform of a Gaussian process, if it makes sense. For that purpose, let me describe the following situation. Let $z(s)$ be a ...
131 views

Understanding kernel PCA when the target space is infinite-dimensional

The PCA optimization problem is known as $$\max_{U \in \mathbb{R}^{d\times r}, U^TU = I} tr(U^T\Sigma U),$$ where $\Sigma$ is a covariance matrix of a dataset $\{x_1,\dots,x_n\} \subset \mathbb{R}^d$...
149 views

Are null space of matrix and kernel function same?

I have recently started learning about machine learning and have come across kernels and null spaces. I understand that null space is the set of all vectors that satisfy the equation A.v = 0 (Where A ...
24 views

kernel publications

What are some contemporary papers that provide the reader with a complex overview of kernel functions used nowadays in machine learning?
37 views

eigenvalue perturbation theory for kernel function

Let $S=\{x_i\}_{i=1}^n$ be a set of training examples, and let $K\in \mathbb{S}^n_+$ be the kernel matrix induced by $S$ and some kernel function $k$ (i.e., $K_{ij}=k(x_i,x_j)$). I was wondering how ...
371 views

How to calculate RKHS norm of a function under given kernel transformation

This was a question asked before in mathoverflow but not yet got answered. I have the same problem when reading Srinivas et al (2010) [appendix B]'s paper. Here are my problems: Definitions: ...
211 views

24 views

Does there exist a vector Ī± st the equality holds?

A training set $(x_1,y_1),...,(x_m,y_m)$ is generic iff $x_i=x_j$ then $y_i=y_j$ and let's consider the following kernel $K_a(x,t)=\prod_{i=1}^n(1+(x_it_i)+(1-x_i)(1-t_i))$ Given a generic training ...
55 views

The trace term in 2 Wassersteins metric for Gaussians

I was looking at the formula for 2 Wassersteins distance for Gaussian distribution on Wikipedia. https://en.wikipedia.org/wiki/Wasserstein_metric#Normal_distributions It satisfies all properties of a ...
197 views

addition of a kernel and a function

I have this problem. My approach would be to take a Kernel matrix of $\ k(x,y)$ as it is positive semidefinitive. And if I am able to make $\ k'(x,y)$ a kernel matrix, then I can prove that it ...
554 views

How to understand effect of RBF kernel for kernel PCA

I understand the math in kernel PCA and with RBF kernel, and I also understand that the RBF kernel map the data into a infinite dimensional space. I know that for SVM, mapping the data into a higher ...
57 views

137 views

139 views

1k views

Proof that $K(x,y) = f(x)f(y)$ is a kernel

I cannot find a proof that $K(x,y)=f(x)f(y)$ is a Kernel for any real valued function $f$ and $x,y\in \mathbb{R}^n$. Obviously it is symmetric but can someone prove that it is positive semidefinite? ...
3k views

Proof that exponential of a kernel is a kernel

How can I prove that the exponential $\exp(K)$ of a kernel function $K$ is again a kernel? I think it can be proved using Taylor expansion but I am not sure how.
1k views

Sample from Gaussian Process across 2D

This seems like a straightforward questions; my apologies if it is already answered (I have looked). Problem: I would like to sample from a Gaussian Process (GP) prior over X and Y coordinates (e.g. ...
576 views

Calculation of nu and gamma in one-class SVM with rbf kernel

I am using python sklearn's one-class svm classifier for anomaly detection. I would like to know can I accurately calculate the required value for nu and gamma for rbf kernel. Is there any equation or ...
4k views

How do I prove such a kernel is positive semi definite? K(x, y) = min(x, y) - xy over [0, 1]

For such a kernel: K(x, y) = min(x, y) - xy over [0, 1] X [0, 1] How can I prove that it's positive semi definite? I know how to prove ...
I am trying to find an intuition on why we require that kernels are positive semi definite and I have found this: We are given a dataset $X$ of size $n \times d$ where $n$ is the number of samples ...