Kernel trick refers to kernel methods in machine learning, such as kernel support vector machine (SVM) or kernel principal components analysis (PCA). It allows to generalize linear techniques to non-linear situations. DO NOT USE this tag for [kernel] which is reserved for non-parametric estimation ...

learn more… | top users | synonyms

0
votes
0answers
6 views

SVM with pre-computed kernel and zero bias

I have an optimization function, where I need to give my own kernel matrix and bias value is zero. The kernel matrix is calculated using the data but there is no specific formula for it. If I have a ...
0
votes
0answers
7 views

LIBSVM for pre-computed kernel and zero bias (b values is zero)

I want to do binary classification and I'm using LIBSVM library for that. I have a precomputed Kernel and my bias value (b) is zero. Can I do this in LIBSVM or do I have to use some other library? ...
0
votes
0answers
13 views

Usage of libsvm with RBF kernel and no Offset

I'm using libsvm for the binary classification and using a precomputed Kernel. In my particular problem there is no bias term (it's zero). Is there anyway to adjust the bias term in libsvm (and not ...
0
votes
1answer
16 views

What is the toolkit that implements Cost sensitive Support Vector Machine?

I need implementation of cost sensitive support vector machine. The cost is different for each training example (unlike each class). So problem is to solve $max_\alpha$ $-1/2 \sum_{i,j} ...
1
vote
1answer
71 views

Which PCA (or kernel PCA) basis better describes a single test sample?

I have two PCA bases obtained by decomposition of two groups of training data. I also have some samples of test data. How can I decide which PCA basis fits better each test sample? I tried to ...
0
votes
0answers
13 views

definig distances using radial basis functions

in svm the kernels are supposed to measure the distance between two vectors in the feature space. however, rbf is largest at 0 meaning that in that new space the distance between a feature and itself ...
2
votes
1answer
39 views

Support vector regression versus kernel ridge regression

I have a question concerning the difference between support vector regression and kernel regression. I will try to write down all the math so no misunderstandings arise (hopefully). Let's begin with ...
4
votes
1answer
124 views

How to apply a Gaussian radial basis function kernel PCA to nonlinear data?

I have an assignment to implement a Gaussian radial basis function-kernel principal component analysis (RBF-kernel PCA) and have some challenges here. It would be great if someone could point me to ...
2
votes
1answer
30 views

Are eigenvectors obtained in Kernel PCA orthogonal?

As Kernel PCA is the same as PCA in higher dimension space, shouldn't the eigenvectors obtained be orthogonal? Suppose, I have $n$ data points and let $a$ and $b$ be two eigenvectors of covariance ...
1
vote
1answer
76 views

Support Vector Machine Question

I need help with the following problem. I provided my current (partial) solution, and I hope someone can correct me and/or give me suggestions as to how I should solve the parts that I've left out. ...
1
vote
1answer
61 views

Where can I use kernels other than Gaussian (like Cauchy, laplacian) in kernel methods in machine learning? Or maybe in kernel density estimation?

In few papers I read that - kernel used doesn't really matter for kernel density estimation but bandwidth of the kernel is the most important factor. But I did not see any mathematical explanation to ...
0
votes
0answers
5 views

How accurate sum of kernel function needs to be, so that we can use it in Mean shift algorithm (may be for image segmentation)?

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. It is useful for detecting the modes of this density. This is an iterative ...
1
vote
2answers
268 views

Plotting the decision boundary of a kernel SVM (RBF)

Suppose we are given a training set of 2D points that are linearly non-separable. I train a binary SVM with an RBF kernel in order to classify them. What I want to do is to draw the desicion boundary. ...
4
votes
3answers
109 views

Do kernel methods “scale” with the amount of data?

I've been reading about kernel methods, where you map original $N$ data points to a feature spaces, compute the kernel or gram matrix and plug that matrix into a standard, linear algorithm. This all ...
4
votes
2answers
211 views

Is Kernel PCA with linear kernel equivalent to standard PCA?

If in kernel PCA I choose a linear kernel $K(\mathbf{x},\mathbf{y}) = \mathbf x^\top \mathbf y$, is the result going to be different from the ordinary linear PCA? Are the solutions fundamentally ...
3
votes
1answer
97 views

Understand the reasons of using Kernel method in SVM

I understand that one can use kernel functions (i.e. radial kernel) to create non-linear decision boundary. However, there is something with my logic and I am sure there is something that I clearly ...
0
votes
0answers
108 views

What's wrong with my Kernel algorithm (Kernel SVD)?

I have a user-item matrix $A$ as data input, which is a sparse matrix containing a large number of missing values (as zeros). Each row is a user, and each column is an item. Generally, I am conducting ...
3
votes
1answer
194 views

What are the advantages of kernel PCA over standard PCA?

I want to implement an algorithm in a paper which uses kernel SVD to decompose a data matrix. So I have been reading materials about kernel methods and kernel PCA etc. But it still is very obscure to ...
1
vote
1answer
63 views

Linear Kernel in Baysian Linear Regression

I came up with http://mlg.eng.cam.ac.uk/duvenaud/cookbook/index.html and it is actually very useful. At some point it says If you use just a linear kernel in a GP, you're simply doing Bayesian ...
1
vote
0answers
67 views

Derive squared exponential covariance function

In Gaussian Processes, SVMs, kernels are used (as to my understanding) as similarity measure. However, they have the constraint that any kernel has to be represented as a dot product. i.e. ...
0
votes
1answer
39 views

What's wrong with the intuition that kernel measures similarity between observations?

Near the middle of page 16 of Andrew Ng's notes on SVM, he explained an intuitive view of kernel as measuring similarity between observations, but then added the caveat that there are things wrong ...
0
votes
0answers
43 views

KPCA in R proportion of variability explained

I'm using kpca function from kernlab and try to get the proportion of variance explained by each component as in standard pca. I ...
2
votes
2answers
331 views

The difference of kernels in SVM?

Can someone please tell me the difference between the kernels in SVM: Linear Polynomial Gaussian (RBF) Sigmoid Because as we know that kernel is used to mapped our input space into high ...
2
votes
2answers
73 views

Is this a decent summary of the kernel trick?

Here's my understanding of the kernel trick. The motivation is to find a linear separator in a higher dimensional space than what you have (because the data are not currently linearly separable.) ...
9
votes
3answers
95 views

Should I use the Kernel Trick whenever possible for non-linear data?

I recently learned about the use of the Kernel trick, which maps data into higher dimensional spaces in an attempt to linearize the data in those dimensions. Are there any cases where I should avoid ...
0
votes
0answers
201 views

Explicit mapping of input space to a high dimensional feature space

I am now studying on Kernel Methods, all the theory behind it to understand Support Vector Machines. Of course, I understood some very well but there is something I could not completely comprehend. ...
1
vote
1answer
130 views

Kernel PCA with an SVD algo

Suppose that I have a great algo for calculating the SVD and I want to do Kernel PCA. It is possible to first apply the Kernel function to my data and then run the SVD algo on the transformed data?
3
votes
0answers
74 views

Kernel interpretation of Manhattan distance / Lp norms

Euclidean distance corresponds to the linear kernel similarity via: $$d_\text{Euclidean}(x,y)^2=\sum_i (x_i-y_i)^2=\sum_i x_i^2 + y_i^2 - 2 x_i y_i = ...
1
vote
2answers
154 views

Applying kernel function to input data before giving it to algorithm

I have gene expression data, I do dimensionality reduction and clustering with self organizing maps, but self organizing maps do not perform well with my data. I want to map my data to feature space ...
1
vote
1answer
139 views

Non-Orthogonality in PCA?

i) What is the main role of "only" trying to find orthogonal components in PCA? I can understand, that we would not want a zero-solution as well as find directions that are orthogonal in order to ...
7
votes
1answer
146 views

Kendall-tau and RKHS spaces

Given two random variables $X_1$ and $X_2$, the Kendall-tau correlation coefficient could be defined as $$ ...
0
votes
1answer
56 views

is it possible to use kernel function to calculate each instance of covariance matrix? If Yes why?

I saw a paper that uses Gaussian kernel for calculation at covariance matrix of given variables. Is it mathematically Correct or not? if it is okay,what is the intuition behind ? What about using any ...
1
vote
0answers
27 views

Confusion related to convexity of a function

I was reading this paper http://www.umiacs.umd.edu/~jags/pdfs/KSforNLP.pdf where they say that $trace(K \pi^{T} L \pi)$ is a convex function of $\pi$ where $\pi$ is a permutation matrix and K and L ...
2
votes
0answers
147 views

Generalized RBF Kernels

There is the notion of Generalized RBF Kernels, for example in "Towards Optimal Bag-of-Features for Object Categorization and Semantic Video Retrieval" from Jiang (1) or in formula (2.72) in ...
1
vote
1answer
1k views

Use Gaussian RBF kernel for mapping of 2D data to 3D

I am working on SVMs and try to get all the concepts involved. For instance, the kernel mapping. I would like to construct some parts of the algorithm by myself, to understand what is happening. My ...
1
vote
1answer
208 views

Prove (or disprove) that this function is a kernel

I devised a distance function similar to this form $d(x,y) = \sum_{i = 1}^{n-1} b(x_i, y_i,x_{i+1}, y_{i+1}) $ with $b(x_i, y_i,x_{i+1}, y_{i+1}) = 0 \mbox{ if } x_i \leq 0 \vee y_i \leq 0 \vee ...
2
votes
1answer
608 views

Kernel PCA (in R)

I am attempting to use the kernel PCA features in kernlab but am having trouble understanding the output. In particular, it's unclear what scale the results are in ...
1
vote
1answer
214 views

Number of kernel evaluations in SVM training

What is the typical number of kernel evaluations (between two training vectors) performed during a (kernelized) Support Vector Machine (SVM) training? I am asking this question because I need to ...
1
vote
0answers
142 views

How to choose kernel functions for support vector regression

Are there any good resources regarding how to design kernels for regression problems, specifically time-series regression type of problem. I am finding the choice of a kernel for regression extremely ...
0
votes
0answers
129 views

Does kernel trick correspond to real higher dimensions?

I've read that the SVM kernel tricks corresponds to viewing the data in a higher dimension. I can see that the expansion of the kernel looks like a cross-product of many different terms. However, it ...
7
votes
1answer
468 views

Kernel Ridge Regression Efficiency

Ridge Regression can be expressed as $$\hat{y} = (\mathbf{X'X} + a\mathbf{I}_d)^{-1}\mathbf{X}x$$ where $\hat{y}$ is the predicted label, $\mathbf{I}_d$ the $d \times d$ identify matrix, $\mathbf{x}$ ...
1
vote
1answer
338 views

Proof of closeness of kernel functions under pointwise product

How can I prove that pointwise product of two kernel functions is a kernel function?
8
votes
1answer
1k views

What function could be a kernel?

In the context of machine learning and pattern recognition, there's a concept called Kernel Trick. Facing problems where I am asked to determine whether a function could be a kernel function or not, ...
7
votes
1answer
2k views

What are the advantages of Multiple Kernel Learning (MKL) methods?

Multiple Kernel Learning methods aim to construct a kernel model where the kernel is a linear combination of fixed base kernels. Learning the kernel then consists of learning the weighting ...
0
votes
1answer
108 views

SVM decision function

our decision function e.g. in SVMs for binary classification (where the response is labeld by $y_i \in \{-1,1\}$) has the form: $f(\mathbf{x}) = \text{sgn}(\mathbf{w}^\top \mathbf{x} + b)$ where ...
1
vote
1answer
96 views

Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen?

Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen? If so, to what dimension does it transform the data?
3
votes
1answer
658 views

Kernelised k Nearest Neighbour

I'm new to kernels and have hit a snag while trying to kernelise kNN. Preliminaries I'm using a polynomial kernel: $K(\mathbf{x},\mathbf{y}) = (1 + \langle \mathbf{x},\mathbf{y} \rangle)^d$ Your ...
1
vote
1answer
1k views

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function: $$ ...
0
votes
1answer
364 views

Free data set for comparing kernel logistic regression and regular logistic regression [closed]

I'm looking for a data set that is easily accessible for comparing Kernel Logistic Regression (KLR) and regular logistic regression. All the paper that I find using KLR use synthetic data sets. I'm ...
2
votes
0answers
92 views

Polynomial kernel in logistic regression?

So I have put together a nice logistic regression program that works quite well. Now, I have used two dimensions to test it and see how it works, and guided by some online tutorials, have increased ...