Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

learn more… | top users | synonyms

2
votes
0answers
13 views

Kernel function between time series of different lengths

I'm studying a data set composed of time series of different lengths; some are up to an order of magnitude longer than others. (If it matters, the data aren't actually temporally related; it's just ...
0
votes
0answers
11 views

How can you scale Gram matrix for SVM classification?

Why scaling is important for the linear SVM classification? gives a good intuition on why scaling is important when using SVM classification. There are two types of input when using SVM fit method : ...
0
votes
0answers
8 views

0-centering a kernel matrix: which formula is the correct one?

I'm trying to implement the algorithm in this paper. As you can see at page 4, it is explained how to 0-centering a pxp symmetric Kernel matrix ...
3
votes
1answer
53 views

Eigenvalue equation for kernel PCA

In Nonlinear component analysis as a kernel eigenvalue problem, Schölkopf et al start by describing PCA. Given a set of data instances $x_1, \dots, x_M$, with $x_k \in \mathbb{R}^N, k=1,\dots,M$, and ...
1
vote
0answers
14 views

Chi square kernel function: which version?

I'm not a machine learning expert, so sorry if this is a stupid question. I was reading this paper about Kernenlized locality-sensitive hashing (LSH) and in section 6.4 the chi-square-kernel function ...
2
votes
0answers
21 views

What does the term data dependent kernel mean?

I was reading a paper that revisits (singled layered) Radial Basis Function networks. In they claim that they show that RBF methods can be recast as "certain data-dependent kernels". I was wondering ...
3
votes
0answers
39 views

Rank of kernel Gram matrix and classifier performance

In kernel machines we have some kernel function $k$ and we compute the $n \times n$ Gram matrix $K$ where $K_{ij} = k(x_i, x_j)$ for observations $x_i, x_j \in \mathbb R^p$. I'm letting $n$ denote the ...
1
vote
1answer
21 views

Same kernel for mixed/categorical data?

I know it's common practice, but is it right to apply the common kernels to categorical/mixed data? If not, are there alternatives? I'm expecting answers from both theoretical and practical points of ...
0
votes
0answers
17 views

Mercer's positive semidefinite kernels?

I have just come across a surprising statement in a highly cited paper. Discussing SVMs, the authors say Note that for the commonly-used Mercer kernels, [the Gram matrix] $G$ is a symmetric ...
1
vote
0answers
18 views

Projection on weighted kernel PCA basis

I'm performing a sort of weighted kernel PCA, where the weights of samples can be negative. The weights of all samples are given by the diagonal weight matrix $D$. The data matrix is the $n \times d$ ...
2
votes
1answer
25 views

What will be the input value (i.e. $x$ and $x′$) of RBF kernel for a given dataset or data matrix $x$?

If $x$ is a data matrix or dataset then What will be the input value (i.e. $x$ and $x'$) of RBF kernel $K_r(x,x')=\exp(-\frac{\|x-x'\|^2}{r})$ ? I can understand $x$ is same as dataset or data matrix ...
0
votes
1answer
12 views

Kernel feature mapping: Derivation of polynomial kernel

The question is related to the derivation shown in section 3.1 Examples in the following lecture: http://people.eecs.berkeley.edu/~jordan/courses/281B-spring04/lectures/lec4.pdf I am confused about ...
1
vote
0answers
17 views

Is it possible to yield high-dimensional data from its low-dimensional point in KPCA?

With PCA, it is possible to reconstruct high-dimensional data from its low-dimensional point by $$ x_i' = Pb + \bar X $$ Where $\bar X$ is the mean of training set $X$, $P$ is the eigenvectors and $b$ ...
2
votes
1answer
41 views

What is the definition of a kernel on vertices or edges?

I am currently trying to perform clustering on a collection C of undirected and unlabeled graphs. I decided to use to a kernel on graphs to obtain the kernel matrix of C. Then I can derive the ...
1
vote
0answers
15 views

Reference request about feature maps in ML

Can someone kindly link to some recent papers on understanding feature maps in ML? It would help to get an idea of what are the recent issues there that people have been working on with regards to ...
0
votes
1answer
32 views

How to justify the usage of “Radial Basis Function” as a kernel for SVM [duplicate]

I ran SVM with several kernels on my data. RBS has the best performing results. The task is similar to the text classification. I wonder how I can explain why RBS is actually the best kernel for my ...
3
votes
1answer
44 views

Why must kernel functions be scalar products [duplicate]

I'm currently reading Bishop's Pattern Recognition and Machine Learning. In the chapter on kernel methods, he's very clear that kernels must be "valid", that is: be representable as scalar products in ...
0
votes
0answers
28 views

Does it make sense to do PCA before kernel regression?

I have a set of features extracted from the same samples and I'm learning a kernel ridge regression. Now, especially for feature fusion, reducing the number of features before combining them seems ...
1
vote
0answers
10 views

Is it needed to regularize in case you know your data is generated by a model of your model class?

Assume we have a dataset $X_{full}$ with labels $y_{full}$. We train a kernel ridge regression model on this data with the Gaussian kernel. This model is used to generate predictions on the whole ...
0
votes
1answer
64 views

How does gamma in SVM RBF kernel influence the accuracy?

I am working on a classification program using SVM RBF kernel. To find the best parameters C and gamma, I used grid search, and got the image below. What confuses me is that when gamma varies from 0.3 ...
0
votes
0answers
11 views

What happens to the RKHS of a Gaussian kernel if $\sigma$ is increased / decreased?

I'm especially interested in the following case: let's say we have a Gaussian kernel $K$ with bandwidth $\sigma$ and RKHS $\mathcal{H}$ and a set $H = \{ \forall h \in \mathcal{H} : ||h||_K \leq \...
0
votes
0answers
36 views

Gaussian kernel cosine distance

I have a dataset with clustering results from $K$=3 to 70 clusters for $n$ observations. My original dataset is an $n \times m$ matrix with $n$ rows (one for each observation) and $m$ features. I am ...
0
votes
0answers
10 views

How to find a similarity score of objects based on multiple nominal/ordinal/continuous variables

I am trying to solve a problem for finding similarity score between objects to create a similarity score matrix based on multiple nominal/ordinal/continuous variables for each object. Example of how ...
5
votes
2answers
77 views

Is Gradient Descent possible for kernelized SVMs (if so, why do people use Quadratic Programming)?

Why do people use Quadratic Programming techniques (such as SMO) when dealing with kernelized SVMs? What is wrong with Gradient Descent? Is it impossible to use with kernels or is it just too slow (...
1
vote
0answers
7 views

Circularity of Gaussian kernels and infinite dimensionality [closed]

I'd like to know more about this statement: Gaussian kernels are circular (which leads to the above-mentioned infinite dimensionality?) Questions: in image processing the issue of ...
0
votes
0answers
31 views

What is the problem with negative eigenvalue (in gram-matrix) in SVM?

This is probably very basic, but I still don't know the answer. I am working on homework in my course, and one of the questions is dealing with negative eigenvalue in gram matrix at SVM, can someone ...
0
votes
0answers
48 views

Creating a Radial basis function kernel matrix in matlab

I never used matlab, and I have this code about kernalized locality sensitive functions. I think that the following code is trying to create the kernalized matrix of a RBF kernel function: ...
3
votes
1answer
92 views

Learning functional analysis for studying kernels

I'm trying to learn more about kernel machine theory and I've discovered that I need to learn a lot of background math, and so I'm looking for some good resources for this. In particular: I've got ...
1
vote
0answers
16 views

Are VC dimensionality and dimension of kernel used in SVM related?

Are VC dimensionality and dimension of kernel used in SVM related to each other? or are they independent parameters in a classification process ?
0
votes
0answers
10 views

Representer Theorem: Is the RKHS $H$ the feature space or the estimator space?

I don't know how theoretical this page is, but I don't know a better place to ask this. It is more an intuitive question than a formal one. In reproducing kernel hilbert space theory we normally have ...
-1
votes
1answer
32 views

Why training error > 0 on SVM with RBF kernel

When using RBF kernel I think feature space is infinite dimensional space. With infinite dimensional features, I believe any training set can be classified. So I'm wondering why training error > 0 ...
1
vote
0answers
36 views

Understanding Kernel Ridge Regression and How It Works (and Implementing it in R)

I am trying to understand how KRR works for drug-protein-interaction and many aspects of it seem very confusing. Supposing I have a data set as follows of Drug-Protein interactions; values show how ...
0
votes
0answers
24 views

Parameter ranges for sigmoid and polynomial kernel

I would like to use a SVM classifier with a sigmoid and polynomial kernel. The sigmoid kernel has the following form: $K(u,v) = \tanh(\gamma * u'v + \text{coef}_{0})$ The polynomial kernel has the ...
1
vote
0answers
9 views

Are there kernel-based one-class sparse kernel-based outlier detection methods, e.g. one-class Relevance Vector Machine?

I have a commercial outlier detection problem in moderate dimension (8-25). We have a limited number of true positive tags and can roughly evaluate performance of various methods. So far, the 1-...
0
votes
0answers
15 views

Two ways of defining a squared kernel: are they equal?

Let's say I have a PDS kernel $K$, and a corresponding featuremap $\psi(x)$. I am interested in vectors in the RKHS / featurespace of $K$ with limited norm, the norm is limited by $\Lambda$. Now I ...
0
votes
1answer
27 views

What does the number 'Kernel Option' refer to in SVM?

I read that the performance of some kernel functions in SVM can change if we change the number known as kernel option. For example, this article states that kernel option of value 2 was used, http://...
0
votes
0answers
23 views

The feature space from Gaussian kernel is infinite-dimensional, are there countably or uncountably many basis?

My attempt: Let $x,y\in\mathbb{R}^d$. We already know the Fourier transform of a Gaussian function is a Gaussian function.If substituting $x-y$ for the variable after Fourier transform, we have $$ \...
0
votes
0answers
11 views

Difference between polynomial regression and polynomial kernel

A few answers on SO suggested that a polynomial transformation and a regularized regression can be used instead of a polynomial kernel regression. What's the difference between them? I thought ...
1
vote
1answer
27 views

Is a kernel space necessarily high dimensional?

It's very possible that I'm misunderstanding one or more terms here, so I will just try to explain what I understand (and why it doesn't make sense) :) Say I have an $N$x$D$ data matrix, i.e. $N$ ...
1
vote
0answers
24 views

Even property of Kernel in Non-parametric statistics

In Non-parametric statistics one requirement for the kernel is : $$ K(-u)=K(u) $$ for all values of $u$. This requirement ensures that the average of the corresponding distribution is equal to that of ...
0
votes
0answers
9 views

How to extract PCs in kpca

I am using RBF kernel in KPCA of Kernlab. What is the procedure for extracting the k pcs which explain the maximum variance. In KPCA, I learned that the PCA will be done in a higher dimensional space,...
0
votes
0answers
19 views

Spark big matrix computation

I am trying to see if Spark is fit for my problem. I would like to use kernel methods, as presented in Kernel method. For example, kernel k-means. One important feature of the kernel methods is that ...
4
votes
1answer
61 views

What's the physical meaning of the eigenvectors of the Gram/Kernel matrix?

If we have some centered dataset $X$ then the eigenvectors of $X^TX$ represent the principal components of the dataset, and their physical meaning is the directions that data follow in the original ...
0
votes
1answer
37 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
5
votes
1answer
62 views

Geometric Interpretation of Whether SVMs are performing well or not

I came across this research paper which contained this figure which talks about the center of mass (presumably, of the training dataset's datapoints?) and represents the solution of an SVM as ...
1
vote
2answers
119 views

Projecting to lower/higher-dimensional space for classification: dimensionality reduction vs kernel trick

Whilst learning about classification, I have seen two different arguments. One is that projecting the data to a lower-dimensional space, such as with PCA, makes the data more easily separable. The ...
2
votes
2answers
62 views

Understanding Kernel Functions for SVMs

I am learning about Support Vector Machines, and in particular, those with kernels for non-linear decision boundaries. I understand the concept of projecting the original data to a higher-dimensional ...
1
vote
1answer
42 views

What does a kernel function do in English

The kernel trick avoids the explicit mapping that is needed to get linear learning algorithms to learn a nonlinear function or decision boundary. For all and in the input space , certain functions can ...
1
vote
1answer
172 views

On the properties of Hyperbolic Tangent Kernel

I've read from various sources that Hyperbolic Tangent kernels are not positive semi-definite and thus are not actually a valid kernel. Does this mean they are misnomer? Furthermore, if they are ...
1
vote
1answer
40 views

How are Hyperplane Heatmaps created and how should they be interpreted?

For nonlinear data, when we are using Support Vector Machines, we can use kernels such as Gaussian RBF, Polynomial, etc to achieve linearity in a different (potentially unknown to us) feature space ...