Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

learn more… | top users | synonyms

0
votes
0answers
26 views

What is the problem with negative eigenvalue (in gram-matrix) in SVM?

This is probably very basic, but I still don't know the answer. I am working on homework in my course, and one of the questions is dealing with negative eigenvalue in gram matrix at SVM, can someone ...
0
votes
0answers
11 views

Creating a Radial basis function kernel matrix in matlab

I never used matlab, and I have this code about kernalized locality sensitive functions. I think that the following code is trying to create the kernalized matrix of a RBF kernel function: ...
3
votes
1answer
67 views

Learning functional analysis for studying kernels

I'm trying to learn more about kernel machine theory and I've discovered that I need to learn a lot of background math, and so I'm looking for some good resources for this. In particular: I've got ...
1
vote
0answers
12 views

Are VC dimensionality and dimension of kernel used in SVM related?

Are VC dimensionality and dimension of kernel used in SVM related to each other? or are they independent parameters in a classification process ?
0
votes
0answers
7 views

Representer Theorem: Is the RKHS $H$ the feature space or the estimator space?

I don't know how theoretical this page is, but I don't know a better place to ask this. It is more an intuitive question than a formal one. In reproducing kernel hilbert space theory we normally have ...
-1
votes
1answer
28 views

Why training error > 0 on SVM with RBF kernel

When using RBF kernel I think feature space is infinite dimensional space. With infinite dimensional features, I believe any training set can be classified. So I'm wondering why training error > 0 ...
1
vote
0answers
28 views

Understanding Kernel Ridge Regression and How It Works (and Implementing it in R)

I am trying to understand how KRR works for drug-protein-interaction and many aspects of it seem very confusing. Supposing I have a data set as follows of Drug-Protein interactions; values show how ...
0
votes
0answers
17 views

Parameter ranges for sigmoid and polynomial kernel

I would like to use a SVM classifier with a sigmoid and polynomial kernel. The sigmoid kernel has the following form: $K(u,v) = \tanh(\gamma * u'v + \text{coef}_{0})$ The polynomial kernel has the ...
1
vote
0answers
7 views

Are there kernel-based one-class sparse kernel-based outlier detection methods, e.g. one-class Relevance Vector Machine?

I have a commercial outlier detection problem in moderate dimension (8-25). We have a limited number of true positive tags and can roughly evaluate performance of various methods. So far, the ...
0
votes
0answers
15 views

Two ways of defining a squared kernel: are they equal?

Let's say I have a PDS kernel $K$, and a corresponding featuremap $\psi(x)$. I am interested in vectors in the RKHS / featurespace of $K$ with limited norm, the norm is limited by $\Lambda$. Now I ...
0
votes
1answer
24 views

What does the number 'Kernel Option' refer to in SVM?

I read that the performance of some kernel functions in SVM can change if we change the number known as kernel option. For example, this article states that kernel option of value 2 was used, ...
0
votes
0answers
22 views

The feature space from Gaussian kernel is infinite-dimensional, are there countably or uncountably many basis?

My attempt: Let $x,y\in\mathbb{R}^d$. We already know the Fourier transform of a Gaussian function is a Gaussian function.If substituting $x-y$ for the variable after Fourier transform, we have $$ ...
0
votes
0answers
8 views

Difference between polynomial regression and polynomial kernel

A few answers on SO suggested that a polynomial transformation and a regularized regression can be used instead of a polynomial kernel regression. What's the difference between them? I thought ...
1
vote
1answer
25 views

Is a kernel space necessarily high dimensional?

It's very possible that I'm misunderstanding one or more terms here, so I will just try to explain what I understand (and why it doesn't make sense) :) Say I have an $N$x$D$ data matrix, i.e. $N$ ...
1
vote
0answers
24 views

Even property of Kernel in Non-parametric statistics

In Non-parametric statistics one requirement for the kernel is : $$ K(-u)=K(u) $$ for all values of $u$. This requirement ensures that the average of the corresponding distribution is equal to that of ...
0
votes
0answers
9 views

How to extract PCs in kpca

I am using RBF kernel in KPCA of Kernlab. What is the procedure for extracting the k pcs which explain the maximum variance. In KPCA, I learned that the PCA will be done in a higher dimensional ...
0
votes
0answers
17 views

Spark big matrix computation

I am trying to see if Spark is fit for my problem. I would like to use kernel methods, as presented in Kernel method. For example, kernel k-means. One important feature of the kernel methods is that ...
4
votes
1answer
52 views

What's the physical meaning of the eigenvectors of the Gram/Kernel matrix?

If we have some centered dataset $X$ then the eigenvectors of $X^TX$ represent the principal components of the dataset, and their physical meaning is the directions that data follow in the original ...
0
votes
0answers
27 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
5
votes
1answer
62 views

Geometric Interpretation of Whether SVMs are performing well or not

I came across this research paper which contained this figure which talks about the center of mass (presumably, of the training dataset's datapoints?) and represents the solution of an SVM as ...
1
vote
2answers
74 views

Projecting to lower/higher-dimensional space for classification: dimensionality reduction vs kernel trick

Whilst learning about classification, I have seen two different arguments. One is that projecting the data to a lower-dimensional space, such as with PCA, makes the data more easily separable. The ...
2
votes
2answers
54 views

Understanding Kernel Functions for SVMs

I am learning about Support Vector Machines, and in particular, those with kernels for non-linear decision boundaries. I understand the concept of projecting the original data to a higher-dimensional ...
1
vote
1answer
38 views

What does a kernel function do in English

The kernel trick avoids the explicit mapping that is needed to get linear learning algorithms to learn a nonlinear function or decision boundary. For all and in the input space , certain functions ...
1
vote
1answer
147 views

On the properties of Hyperbolic Tangent Kernel

I've read from various sources that Hyperbolic Tangent kernels are not positive semi-definite and thus are not actually a valid kernel. Does this mean they are misnomer? Furthermore, if they are ...
1
vote
1answer
36 views

How are Hyperplane Heatmaps created and how should they be interpreted?

For nonlinear data, when we are using Support Vector Machines, we can use kernels such as Gaussian RBF, Polynomial, etc to achieve linearity in a different (potentially unknown to us) feature space ...
2
votes
1answer
22 views

Proving the validity of a kernel

How can we prove that the following is a valid kernel? Let $\phi$ be any function on R X R. Define: $K(x,y) = \int{\phi(x,z)\phi(y,z)dz}$ We want to show that $K$ is a valid kernel.
0
votes
1answer
44 views

Why can kernel PCA with Gaussian kernel separate half-moon shapes and concentric circles but not Swiss Roll?

According to this website, kernel PCA with RBF (Gaussian) kernel can separate half-moon shapes and concentric circles effectively but not Swiss Roll shapes (in 3-D). I don't understand why it doesn't ...
0
votes
0answers
17 views

SVM kernels combination

Let's suppose we have classification problem with two classes. $$X^l = (x_i, y_i)_{i=1}^l, Y=\{+1, -1\}\;\; y:X\rightarrow Y$$ $$x_i \in R^n, y_i = y(x_i),\;\; i = 1\dots n$$ One of the most spread ...
4
votes
1answer
62 views

Machine Learning SVM

If one trains a model using a SVM from kernel data, the resultant trained model contains support vectors. Now consider the case of training a new model using the old data already present plus a small ...
0
votes
0answers
28 views

Rules for choosing how much training data one needs to learn a Radial Basis Function (RBF) model?

I was trying to understand how much data I would need compared to the number of parameters (and to have good generalization) when I train a radial basis function (RBF) network on a regression task ...
1
vote
2answers
43 views

Corresponding RKHS of Common Kernels

A kernel, $k(x_1, x_2)$, has the interesting property that it may be represented as the dot product in a reproducing kernel hilbert space (RKHS), $\phi(x_0)\phi(x_1)$. I know that for the gaussian ...
1
vote
1answer
57 views

What happens if you square an RBF kernel function?

Let's say we use a kernel regularization algorithm such as ridge regression to minimize some loss in an RBF kernel: $$\min_{h \in H} \frac{1}{n} \sum_i (h(x_i) - y(x_i))^2 + ||h||^2_K$$ We get some ...
1
vote
0answers
31 views

How to plot ROC for knn (and potentially kernel spectral regression)

I understand how to plot ROC for logistic classifier (like varies the probability cutoff). For KNN, how can I find the ROC? Also, what about kernel spectral regression?
0
votes
0answers
24 views

has anyone tried to use spectrum kernel [duplicate]

Can any one explain me how to use string kernels to quantify the similarity between short texts? thank you. regards
1
vote
1answer
155 views

In Convolutional Neural Networks (CNN), how we can decide number of kernels between input and hidden layer?

I have $32\times32$ input image and $5\times5$ convolution. So in the first hidden layer, the feature map size will be $28\times28$. At this link we can see in C1, the number of feature maps is 4 but ...
0
votes
0answers
16 views

After running kernel PCA on the training data, how to apply it to a new data point? [duplicate]

My data set has a training set of 1000 input with 6 features (data set size is 1000*6). I applied kernel PCA to the data set and reduced the number of features to 3. It means that the dimension of the ...
1
vote
0answers
26 views

Gaussian Process: Parameter of Kernel function

I am quite new to kernel method, I am trying to estimate $y'$ corresponding to $x'$, given [x, y] data. I am using Gaussian Method for analysis with Kernel function: $k(x_1,x_2 ) =p_1\exp\{-p_2(x_1 ...
0
votes
0answers
18 views

Choosing orientation for kernel principal component analysis

I have a matrix with orientation [1500x93]. The 1500 corresponds to 1500 time samples and the 93 corresponds to the X,Y and Z coordinates for 31 markers. I am performing principal component analysis ...
0
votes
0answers
50 views

How to compare PCA with KPCA for dimension reduction?

Both linear principal component analysis (PCA) and kernel principal component analysis (KPCA) are unsupervised dimension reduction methods. I have a dataset with $4000$ training samples and $40000$ ...
1
vote
1answer
45 views

linear kernel SVM

The linear kernel is defined as: $K(x1,x2)=\langle x1,x2\rangle$. I can see that all that this kernel does is to calculate the dot product in the original space of the data. Why is this kernel then ...
0
votes
0answers
48 views

Mercer's condition

I am having a very hard time understanding Mercer kernel. If any sequence of data points $x_1, ... , x_n \in R^d$ and coefficients $c_1, ... , c_n \in R$, satisifies the inequality $\sum^n_{i=1} ...
0
votes
0answers
19 views

SVM - relevance of linear kernel

The linear kernel is of the form K(x1,x2)=$<x1,x2>$. I understand that kernel functions help us compute a dot product in some high dimensional space. In the case of the linear kernel, I see that ...
1
vote
0answers
32 views

Stochastic Differential Equation Interpretation of Squared Exponential Kernel

As far as I understand, many Gaussian Processes can be either described by their corresponding mean and kernel functions or by a stochastic differential equation (SDE). For my purposes it is ...
0
votes
0answers
29 views

Why is the squared exponential kernel so popular?

Be it SVM or GPR is seems like, besides the linear kernel, in the kernel machines community the squared exponential kernel $$k(x,x')=\sigma^2\exp\left((x-x')^2/l\right)$$ with $\sigma>0$ and ...
0
votes
1answer
59 views

Eigenfunctions and eigenvalues of the exponential kernel

What are the eigenfunctions and the eigenvalues of the exponential kernel? The exponential kernel is defined as $$k(x,x')=\sigma^2\exp\left(\frac{||x-x'||}{l}\right)$$ where both $\sigma>0$ and ...
0
votes
0answers
11 views

Prove that $k(x,x') = k_a(x_a, x'_a) + k_b(x_b, x'_b)$ where $x = (x_a,x_b)$ is a kernel

Prove that $k(x,x') = k_a(x_a, x'_a) + k_b(x_b, x'_b)$ where $x = (x_a,x_b)$ is a kernel. I can prove that $k(x,x') = k_1(x,x') + k_2(x,x')$ is a kernel but I cannot see how this can be used to solve ...
0
votes
1answer
56 views

PCA vs. Spectral Clustering with Linear Kernel

Consider a feature vector matrix $X := [x_1 x_2 \dots x_d] \in \mathbb {R}^{n\times d} $ that I hope to use as part of some supervised learning procedure, say, regression. Suppose that also, $d \gg n ...
1
vote
0answers
37 views

Can a SVM with polynomial kernel lead to overfitting?

I'm currently searching for the best set of parameters for a Polynomial Kernel with a grid search. I would like to know if using a high value for Polynomial order (10, 100 or 1000) can lead to ...
1
vote
0answers
30 views

SVM - support vectors on the wrong side of the margin for polynomial kernel

In soft-margin SVM, the constraint in soft-margin (compared to hard-margin) is $0≤α_i≤C$ for some positive constant C. We say that: If $α_i=C$, then the corresponding $x_i$ is on the wrong side of ...
0
votes
0answers
26 views

Rademacher complexity of SVM with kernel in terms of whole Kernel Matrix

http://www.cs.nyu.edu/~mohri/mls/lecture_5.pdf In slide no. 18 here, it is shown that Rademacher complexity of SVM with kernel can be written in terms of trace of the matrix. Are there any other ...