# Tagged Questions

Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

13 views

### Kernel function between time series of different lengths

I'm studying a data set composed of time series of different lengths; some are up to an order of magnitude longer than others. (If it matters, the data aren't actually temporally related; it's just ...
11 views

### How can you scale Gram matrix for SVM classification?

Why scaling is important for the linear SVM classification? gives a good intuition on why scaling is important when using SVM classification. There are two types of input when using SVM fit method : ...
8 views

### 0-centering a kernel matrix: which formula is the correct one?

I'm trying to implement the algorithm in this paper. As you can see at page 4, it is explained how to 0-centering a pxp symmetric Kernel matrix ...
53 views

### Eigenvalue equation for kernel PCA

In Nonlinear component analysis as a kernel eigenvalue problem, Schölkopf et al start by describing PCA. Given a set of data instances $x_1, \dots, x_M$, with $x_k \in \mathbb{R}^N, k=1,\dots,M$, and ...
14 views

### Chi square kernel function: which version?

I'm not a machine learning expert, so sorry if this is a stupid question. I was reading this paper about Kernenlized locality-sensitive hashing (LSH) and in section 6.4 the chi-square-kernel function ...
21 views

### What does the term data dependent kernel mean?

I was reading a paper that revisits (singled layered) Radial Basis Function networks. In they claim that they show that RBF methods can be recast as "certain data-dependent kernels". I was wondering ...
39 views

### Rank of kernel Gram matrix and classifier performance

In kernel machines we have some kernel function $k$ and we compute the $n \times n$ Gram matrix $K$ where $K_{ij} = k(x_i, x_j)$ for observations $x_i, x_j \in \mathbb R^p$. I'm letting $n$ denote the ...
21 views

### Same kernel for mixed/categorical data?

I know it's common practice, but is it right to apply the common kernels to categorical/mixed data? If not, are there alternatives? I'm expecting answers from both theoretical and practical points of ...
17 views

### Mercer's positive semidefinite kernels?

I have just come across a surprising statement in a highly cited paper. Discussing SVMs, the authors say Note that for the commonly-used Mercer kernels, [the Gram matrix] $G$ is a symmetric ...
18 views

### Projection on weighted kernel PCA basis

I'm performing a sort of weighted kernel PCA, where the weights of samples can be negative. The weights of all samples are given by the diagonal weight matrix $D$. The data matrix is the $n \times d$ ...
25 views

### What will be the input value (i.e. $x$ and $x′$) of RBF kernel for a given dataset or data matrix $x$?

If $x$ is a data matrix or dataset then What will be the input value (i.e. $x$ and $x'$) of RBF kernel $K_r(x,x')=\exp(-\frac{\|x-x'\|^2}{r})$ ? I can understand $x$ is same as dataset or data matrix ...
12 views

### Kernel feature mapping: Derivation of polynomial kernel

The question is related to the derivation shown in section 3.1 Examples in the following lecture: http://people.eecs.berkeley.edu/~jordan/courses/281B-spring04/lectures/lec4.pdf I am confused about ...
17 views

### Is it possible to yield high-dimensional data from its low-dimensional point in KPCA?

With PCA, it is possible to reconstruct high-dimensional data from its low-dimensional point by $$x_i' = Pb + \bar X$$ Where $\bar X$ is the mean of training set $X$, $P$ is the eigenvectors and $b$ ...
41 views

### What is the definition of a kernel on vertices or edges?

I am currently trying to perform clustering on a collection C of undirected and unlabeled graphs. I decided to use to a kernel on graphs to obtain the kernel matrix of C. Then I can derive the ...
15 views

### Reference request about feature maps in ML

Can someone kindly link to some recent papers on understanding feature maps in ML? It would help to get an idea of what are the recent issues there that people have been working on with regards to ...
32 views

### How to justify the usage of “Radial Basis Function” as a kernel for SVM [duplicate]

I ran SVM with several kernels on my data. RBS has the best performing results. The task is similar to the text classification. I wonder how I can explain why RBS is actually the best kernel for my ...
44 views

### Why must kernel functions be scalar products [duplicate]

I'm currently reading Bishop's Pattern Recognition and Machine Learning. In the chapter on kernel methods, he's very clear that kernels must be "valid", that is: be representable as scalar products in ...
28 views

### Does it make sense to do PCA before kernel regression?

I have a set of features extracted from the same samples and I'm learning a kernel ridge regression. Now, especially for feature fusion, reducing the number of features before combining them seems ...
10 views

### Is it needed to regularize in case you know your data is generated by a model of your model class?

Assume we have a dataset $X_{full}$ with labels $y_{full}$. We train a kernel ridge regression model on this data with the Gaussian kernel. This model is used to generate predictions on the whole ...
64 views

### How does gamma in SVM RBF kernel influence the accuracy?

I am working on a classification program using SVM RBF kernel. To find the best parameters C and gamma, I used grid search, and got the image below. What confuses me is that when gamma varies from 0.3 ...
11 views

37 views

### Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
62 views

### Geometric Interpretation of Whether SVMs are performing well or not

I came across this research paper which contained this figure which talks about the center of mass (presumably, of the training dataset's datapoints?) and represents the solution of an SVM as ...
119 views

### Projecting to lower/higher-dimensional space for classification: dimensionality reduction vs kernel trick

Whilst learning about classification, I have seen two different arguments. One is that projecting the data to a lower-dimensional space, such as with PCA, makes the data more easily separable. The ...
62 views

### Understanding Kernel Functions for SVMs

I am learning about Support Vector Machines, and in particular, those with kernels for non-linear decision boundaries. I understand the concept of projecting the original data to a higher-dimensional ...
42 views

### What does a kernel function do in English

The kernel trick avoids the explicit mapping that is needed to get linear learning algorithms to learn a nonlinear function or decision boundary. For all and in the input space , certain functions can ...
172 views

### On the properties of Hyperbolic Tangent Kernel

I've read from various sources that Hyperbolic Tangent kernels are not positive semi-definite and thus are not actually a valid kernel. Does this mean they are misnomer? Furthermore, if they are ...
40 views

### How are Hyperplane Heatmaps created and how should they be interpreted?

For nonlinear data, when we are using Support Vector Machines, we can use kernels such as Gaussian RBF, Polynomial, etc to achieve linearity in a different (potentially unknown to us) feature space ...