Kernel methods are used in machine learning to generalize linear techniques to nonlinear situations, especially SVMs, PCA, and GPs. Not to be confused with [kernel-smoothing], for kernel density estimation (KDE) and kernel regression.

learn more… | top users | synonyms

0
votes
0answers
23 views

has anyone tried to use spectrum kernel [duplicate]

Can any one explain me how to use string kernels to quantify the similarity between short texts? thank you. regards
0
votes
1answer
17 views

In CNN, How we can decide number of kernels between input and hidden layer?

I have 32x32 input image and 5x5 convolution. So in the first hidden layer feature map size will be 28x28.in this link http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg we can see in C1 no of ...
0
votes
0answers
15 views

After running kernel PCA on the training data, how to apply it to a new data point? [duplicate]

My data set has a training set of 1000 input with 6 features (data set size is 1000*6). I applied kernel PCA to the data set and reduced the number of features to 3. It means that the dimension of the ...
1
vote
0answers
20 views

Gaussian Process: Parameter of Kernel function

I am quite new to kernel method, I am trying to estimate $y'$ corresponding to $x'$, given [x, y] data. I am using Gaussian Method for analysis with Kernel function: $k(x_1,x_2 ) =p_1\exp\{-p_2(x_1 ...
0
votes
0answers
10 views

Choosing orientation for kernel principal component analysis

I have a matrix with orientation [1500x93]. The 1500 corresponds to 1500 time samples and the 93 corresponds to the X,Y and Z coordinates for 31 markers. I am performing principal component analysis ...
0
votes
0answers
31 views

How to compare PCA with KPCA for dimension reduction?

Both linear principal component analysis (PCA) and kernel principal component analysis (KPCA) are unsupervised dimension reduction methods. I have a dataset with $4000$ training samples and $40000$ ...
1
vote
1answer
33 views

linear kernel SVM

The linear kernel is defined as: $K(x1,x2)=\langle x1,x2\rangle$. I can see that all that this kernel does is to calculate the dot product in the original space of the data. Why is this kernel then ...
0
votes
0answers
32 views

Mercer's condition

I am having a very hard time understanding Mercer kernel. If any sequence of data points $x_1, ... , x_n \in R^d$ and coefficients $c_1, ... , c_n \in R$, satisifies the inequality $\sum^n_{i=1} ...
0
votes
0answers
15 views

SVM - relevance of linear kernel

The linear kernel is of the form K(x1,x2)=$<x1,x2>$. I understand that kernel functions help us compute a dot product in some high dimensional space. In the case of the linear kernel, I see that ...
1
vote
0answers
17 views

Stochastic Differential Equation Interpretation of Squared Exponential Kernel

As far as I understand, many Gaussian Processes can be either described by their corresponding mean and kernel functions or by a stochastic differential equation (SDE). For my purposes it is ...
0
votes
0answers
20 views

Why is the squared exponential kernel so popular?

Be it SVM or GPR is seems like, besides the linear kernel, in the kernel machines community the squared exponential kernel $$k(x,x')=\sigma^2\exp\left((x-x')^2/l\right)$$ with $\sigma>0$ and ...
0
votes
1answer
31 views

Eigenfunctions and eigenvalues of the exponential kernel

What are the eigenfunctions and the eigenvalues of the exponential kernel? The exponential kernel is defined as $$k(x,x')=\sigma^2\exp\left(\frac{||x-x'||}{l}\right)$$ where both $\sigma>0$ and ...
0
votes
0answers
10 views

Prove that $k(x,x') = k_a(x_a, x'_a) + k_b(x_b, x'_b)$ where $x = (x_a,x_b)$ is a kernel

Prove that $k(x,x') = k_a(x_a, x'_a) + k_b(x_b, x'_b)$ where $x = (x_a,x_b)$ is a kernel. I can prove that $k(x,x') = k_1(x,x') + k_2(x,x')$ is a kernel but I cannot see how this can be used to solve ...
0
votes
1answer
37 views

PCA vs. Spectral Clustering with Linear Kernel

Consider a feature vector matrix $X := [x_1 x_2 \dots x_d] \in \mathbb {R}^{n\times d} $ that I hope to use as part of some supervised learning procedure, say, regression. Suppose that also, $d \gg n ...
1
vote
0answers
24 views

Can a SVM with polynomial kernel lead to overfitting?

I'm currently searching for the best set of parameters for a Polynomial Kernel with a grid search. I would like to know if using a high value for Polynomial order (10, 100 or 1000) can lead to ...
1
vote
0answers
28 views

SVM - support vectors on the wrong side of the margin for polynomial kernel

In soft-margin SVM, the constraint in soft-margin (compared to hard-margin) is $0≤α_i≤C$ for some positive constant C. We say that: If $α_i=C$, then the corresponding $x_i$ is on the wrong side of ...
0
votes
0answers
15 views

Rademacher complexity of SVM with kernel in terms of whole Kernel Matrix

http://www.cs.nyu.edu/~mohri/mls/lecture_5.pdf In slide no. 18 here, it is shown that Rademacher complexity of SVM with kernel can be written in terms of trace of the matrix. Are there any other ...
1
vote
1answer
23 views

Use RBF kernel with logistic regression?

There are some resources online (e.g. this one) on logistic regression with polynomial kernels, such as $$h_\theta(x)=logistic(\theta_0 + \theta_1x1+ \theta_3x_1^2 + \theta_4x_2^2)$$ I'm wondering ...
4
votes
2answers
100 views

Is there any kernel defined for simplex domain (i.e. probability vector)?

I am wondering if there is any kernel function that is specifically designed for simplex domain. By "simplex domain," I mean a set whose elements are probability vectors. For example, 3-D simplex may ...
1
vote
0answers
32 views

What does the constraint on the signed support vectors in SVMs signify?

What does the constraint $\sum_i \alpha_i y_i = 0$ on the support vectors signify? Does it mean a data set cannot have only one support vector? Can all the support vectors of a data set after ...
2
votes
1answer
29 views

What does prime mean in the notation $k(x,x')$ in the context of kernels?

I admit it's a very specific question, but what is meant by the notation $'$ in Bishop's book Pattern Recognition and Machine Learning? I'd always thought it's a transpose, but this marked by a ...
3
votes
1answer
335 views

Implicit feature space of Power Kernel

For the polynomial kernel, $K(x,y) = (x^Ty+c)^d$, the implicit feature space $\phi$ for which $K(x,y) = \phi(x)^T \phi(y)$ is of finite dimension and well known [1][2]. It is also well known that the ...
3
votes
1answer
53 views

How to prove or disprove this function is valid kernel?

I have the following function $$ K(x, y) = \begin {cases} 1, & if ||x - y||_2 \le 1 \\ 0, & otherwise \end{cases} $$ I'd like to prove (or disprove) that it's a valid kernel function. In ...
1
vote
0answers
122 views

Implementing kernel ridge regression

I want to implement kernel ridge regression in R. My problem is that I can't figure out how to generate the kernel values and I do not know how to use them for the ridge regression. Before going to ...
0
votes
0answers
23 views

VC dimension of SVM with polynomial kernel in $\mathbb{R}^d$?

Following along the lines of the question here: VC dimension of SVM with polynomial kernel in $\mathbb{R^{2}}$ What is the equation for VC-dim of an SVM with a 2nd-degree polynomial kernel ...
1
vote
1answer
20 views

What kernel function (if one exists) is equivalent to adding a feature vector x1^2 + x2^2

Suppose I have a 2d feature set {$\bf x_1, x_2$}. I can create a third feature $\bf x_3 = {x_1}^2 + {x_2}^2$ and train a model on all three features. Is there a kernel function for this ...
0
votes
1answer
40 views

Proof zero extension of kernel is a kernel

I want to prove that the zero extension of a kernel is still a kernel. I.e. for a set of vectors in D: $$k_0(x,x')= \begin{cases}k(x,x') \ if\ x\in D \ and \ x'\in D\\0\ otherwise\end{cases}$$ ...
3
votes
3answers
119 views

Proof that the linear kernel is a kernel, understanding the math

I can't seem to wrap my head around the following proof. I want to show that the linear kernel is a kernel because its Gram matrix is positive semi-definite. There is plenty of information on the ...
2
votes
1answer
60 views

Calibrating a Gaussian Process

In Thoughts on Massively Scalable Gaussian Processes (or any other introduction to Gaussian Processes), authors claim that calibrating a Gaussian process is just maximizing: ...
0
votes
0answers
44 views

Proving valid and invalid kernel

I have searched through a number of tutorials and found that for a kernel to be valid, the kernel matrix must be symmetrical and satisfies Mercer's condition. But however, I could not seem to apply ...
3
votes
0answers
45 views

PLS using a kernel matrix

I would like to use a kernel matrix generated with a custom kernel function to fit a PLS-DA model (I am thinking of caret's PLS-DA at the moment), with only one binary response variable in the Y ...
2
votes
1answer
66 views

How to use “kernel trick” in Stochastic gradient descent?

How to use "kernel trick" in Stochastic gradient descent? I can find kernel perceptron on Wikipedia but I can't find "kernel sgd" anywhere that gives me a clear algorithm to do that. Can someone teach ...
0
votes
0answers
20 views

Kernel and Mercer Kernel

Kernel is a function that takes vectors as input and returns the dot product of the vectors in the feature space (possibility a higher-dimensional space). Mercer's Theorem tells us whether or not a ...
0
votes
0answers
42 views

Defining the proximity between sentences

What are the common ways to define proximity between two sentences ? I did various tries, including: Jaccard Index Cosine similarity Levenshtein distance either using the counts for the Jaccard ...
8
votes
1answer
102 views

How to kernelize a simple perceptron?

Classification problems with nonlinear boundaries cannot be solved by a simple perceptron. The following R code is for illustrative purposes and is based on this example in Python): ...
3
votes
2answers
299 views

Linear combination of two kernel functions

How can I prove that linear combination of two kernel functions is also a kernel function? \begin{align} k_{p}( x, y) = a_1k_1( x, y) + a_2k_2(x,y) \end{align} given $k_1(,)$ and $k_2(,)$ are valid ...
2
votes
1answer
53 views

SVM Kernel confusion

Suppose that we have an array of 10x2 elements (features). Each of these features are two-dimensional. Something like this: ...
1
vote
0answers
47 views

Equivalence between predictive distribution of Bayesian regression and Gaussian process with linear kernel

I am attempting to understand the algebraic relationship between a Gaussian process with linear kernel covariance and a Bayesian regression. I know they are equivalent formulations but I seem to be ...
1
vote
1answer
47 views

Is this a valid kernel?

I just wanted to know whether a kernel could be defined as follows: $$ k(\mathrm{x}, \mathrm{x}') = x_1 + x_2 \quad \mbox{OR} \quad k(\mathrm{x}, \mathrm{x}') = \left<\begin{bmatrix}x_1\\ x_2 ...
0
votes
1answer
87 views

Why is RBF kernel used in SVM?

I learned that due to infinite series expansion of exponential function Radial Basis Kernel projects input feature space to infinite feature space. Is it due to this fact that we use this kernel ...
5
votes
1answer
64 views

What methods exist for tuning graph kernel SVM hyperparameters?

I have some data that exist on a graph $G=(V,E)$. The vertices belong to one of two classes $y_i\in\{-1,1\}$, and I'm interested in training an SVM to distinguish between the two classes. One ...
3
votes
1answer
51 views

Can a pseudo-metric yield a valid kernel?

Disclaimer: This question is somewhat related to: Generalized RBF Kernels. I apologize if this has too much overlap. Say we have some distance $d(x,x')$, where $x$ is from some set $X$. $d(x,x')$ ...
1
vote
2answers
37 views

Are “covariance function” and “kernel function” synonyms?

Are "covariance function" and "kernel function" synonyms? In the Gaussian process (GP) literature, authors typically discuss covariance functions, whereas in the support vector machine (SVM) ...
2
votes
0answers
39 views

Kernel PCA vs principal curve analysis

Both principal curve analysis and kernel PCA provide the ability to find nonlinear PCA. Kernel PCA does this by finding principal components in a higher dimensional space. Principal curve analysis is ...
1
vote
2answers
40 views

How to design a Kernel for Gaussian process that ensure some properties for the function?

I am using gaussian process for regression and i would like to know if there is any way to design a kernel that ensure that my function is always non-negative. All my observables are positive and i ...
3
votes
3answers
333 views

Kernel SVM: I want an intuitive understanding of mapping to a higher-dimensional feature space, and how this makes linear separation possible

I am trying to understand the intuition behind kernel SVM's. Now, I understand how linear SVM's work, whereby a decision line is made which splits the data as best it can. I also understand the ...
0
votes
0answers
51 views

Core vector machine implementation

I came across the following article : http://www.jmlr.org/papers/volume6/tsang05a/tsang05a.pdf, Core Vector Machines: Fast SVM Training on Very Large Data Sets. The approach looks very promising, ...
0
votes
2answers
51 views

SVM classification

I have a small data set of 450 instances with feature vector of 21 feature and and I need to classify (binary classification) I applied Support Vector Machine Kernel Linear and RBF. In my case RBF ...
2
votes
1answer
77 views

Kernel PCA for feature selection for various machine learning algorithms [duplicate]

I would like to forecast stock index returns with SVM, k-NN, and Neural Networks. In advance I want to select my inputs via kernel PCA (kPCA). Everything is performed in R. For the KPCA I use ...
1
vote
1answer
32 views

How to know which Kernel is better?

I am working on an Image recognition software - My first question is since I already explicitly turm my training images to features vector (and also my test images) what is the point of using ...