The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
1answer
38 views

Where can I use kernels other than Gaussian (like Cauchy, laplacian) in kernel methods in machine learning? Or maybe in kernel density estimation?

In few papers I read that - kernel used doesn't really matter for kernel density estimation but bandwidth of the kernel is the most important factor. But I did not see any mathematical explanation to ...
0
votes
0answers
5 views

How accurate sum of kernel function needs to be, so that we can use it in Mean shift algorithm (may be for image segmentation)?

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. It is useful for detecting the modes of this density. This is an iterative ...
1
vote
2answers
58 views

Plotting the decision boundary of a kernel SVM (RBF)

Suppose we are given a training set of 2D points that are linearly non-separable. I train a binary SVM with an RBF kernel in order to classify them. What I want to do is to draw the desicion boundary. ...
4
votes
3answers
91 views

Do kernel methods “scale” with the amount of data?

I've been reading about kernel methods, where you map original $N$ data points to a feature spaces, compute the kernel or gram matrix and plug that matrix into a standard, linear algorithm. This all ...
4
votes
2answers
102 views

Linear PCA versus Linear Kernel PCA

Sources and definitions: PCA: http://en.wikipedia.org/wiki/Principal_component_analysis KPCA: http://en.wikipedia.org/wiki/Kernel_principal_component_analysis My question: If in KPCA i choose a ...
3
votes
1answer
70 views

Understand the reasons of using Kernel method in SVM

I understand that one can use kernel functions (i.e. radial kernel) to create non-linear decision boundary. However, there is something with my logic and I am sure there is something that I clearly ...
0
votes
0answers
58 views

What's wrong with my Kernel algorithm (Kernel SVD)?

I have a user-item matrix $A$ as data input, which is a sparse matrix containing a large number of missing values (as zeros). Each row is a user, and each column is an item. Generally, I am conducting ...
2
votes
0answers
112 views

What are the advantages of kernel PCA over PCA?

I want to implement an algorithm in a paper which uses Kernel SVD to decompose a data matrix. So I have been reading materials about Kernel methods and kernel PCA etc. But it still is very obscure to ...
1
vote
1answer
56 views

Linear Kernel in Baysian Linear Regression

I came up with http://mlg.eng.cam.ac.uk/duvenaud/cookbook/index.html and it is actually very useful. At some point it says If you use just a linear kernel in a GP, you're simply doing Bayesian ...
1
vote
0answers
41 views

Derive squared exponential covariance function

In Gaussian Processes, SVMs, kernels are used (as to my understanding) as similarity measure. However, they have the constraint that any kernel has to be represented as a dot product. i.e. ...
0
votes
0answers
30 views

What's wrong with the intuition that kernel measures similarity between observations?

Near the middle of page 16 of Andrew Ng's notes on SVM, he explained an intuitive view of kernel as measuring similarity between observations, but then added the caveat that there are things wrong ...
0
votes
0answers
25 views

KPCA in R proportion of variability explained

I'm using kpca function from kernlab and try to get the proportion of variance explained by each component as in standard pca. I ...
2
votes
2answers
222 views

The difference of kernels in SVM?

Can someone please tell me the difference between the kernels in SVM: Linear Polynomial Gaussian (RBF) Sigmoid Because as we know that kernel is used to mapped our input space into high ...
2
votes
2answers
63 views

Is this a decent summary of the kernel trick?

Here's my understanding of the kernel trick. The motivation is to find a linear separator in a higher dimensional space than what you have (because the data are not currently linearly separable.) ...
9
votes
3answers
60 views

Should I use the Kernel Trick whenever possible for non-linear data?

I recently learned about the use of the Kernel trick, which maps data into higher dimensional spaces in an attempt to linearize the data in those dimensions. Are there any cases where I should avoid ...
0
votes
0answers
119 views

Explicit mapping of input space to a high dimensional feature space

I am now studying on Kernel Methods, all the theory behind it to understand Support Vector Machines. Of course, I understood some very well but there is something I could not completely comprehend. ...
1
vote
1answer
111 views

Kernel PCA with an SVD algo

Suppose that I have a great algo for calculating the SVD and I want to do Kernel PCA. It is possible to first apply the Kernel function to my data and then run the SVD algo on the transformed data?
3
votes
0answers
63 views

Kernel interpretation of Manhattan distance / Lp norms

Euclidean distance corresponds to the linear kernel similarity via: $$d_\text{Euclidean}(x,y)^2=\sum_i (x_i-y_i)^2=\sum_i x_i^2 + y_i^2 - 2 x_i y_i = ...
0
votes
0answers
39 views

How to project test data onto the principal vectors produced by kpca?

So I have a dataset of dimension 6395x15 and when I apply kernel-PCA on this dataset, I get a rotated matrix of dimesion 6395x596. pcv() gives me the matrix of column-wise eigen-vectors and this ...
0
votes
1answer
119 views

Applying kernel function to input data before giving it to algorithm

I have gene expression data, I do dimensionality reduction and clustering with self organizing maps, but self organizing maps do not perform well with my data. I want to map my data to feature space ...
1
vote
1answer
118 views

Non-Orthogonality in PCA?

i) What is the main role of "only" trying to find orthogonal components in PCA? I can understand, that we would not want a zero-solution as well as find directions that are orthogonal in order to ...
7
votes
1answer
140 views

Kendall-tau and RKHS spaces

Given two random variables $X_1$ and $X_2$, the Kendall-tau correlation coefficient could be defined as $$ ...
0
votes
1answer
50 views

is it possible to use kernel function to calculate each instance of covariance matrix? If Yes why?

I saw a paper that uses Gaussian kernel for calculation at covariance matrix of given variables. Is it mathematically Correct or not? if it is okay,what is the intuition behind ? What about using any ...
1
vote
0answers
25 views

Confusion related to convexity of a function

I was reading this paper http://www.umiacs.umd.edu/~jags/pdfs/KSforNLP.pdf where they say that $trace(K \pi^{T} L \pi)$ is a convex function of $\pi$ where $\pi$ is a permutation matrix and K and L ...
1
vote
0answers
130 views

Generalized RBF Kernels

There is the notion of Generalized RBF Kernels, for example in "Towards Optimal Bag-of-Features for Object Categorization and Semantic Video Retrieval" from Jiang (1) or in formula (2.72) in ...
0
votes
1answer
1k views

Use Gaussian RBF kernel for mapping of 2D data to 3D

I am working on SVMs and try to get all the concepts involved. For instance, the kernel mapping. I would like to construct some parts of the algorithm by myself, to understand what is happening. My ...
1
vote
1answer
178 views

Prove (or disprove) that this function is a kernel

I devised a distance function similar to this form $d(x,y) = \sum_{i = 1}^{n-1} b(x_i, y_i,x_{i+1}, y_{i+1}) $ with $b(x_i, y_i,x_{i+1}, y_{i+1}) = 0 \mbox{ if } x_i \leq 0 \vee y_i \leq 0 \vee ...
2
votes
1answer
540 views

Kernel PCA (in R)

I am attempting to use the kernel PCA features in kernlab but am having trouble understanding the output. In particular, it's unclear what scale the results are in ...
1
vote
1answer
204 views

Number of kernel evaluations in SVM training

What is the typical number of kernel evaluations (between two training vectors) performed during a (kernelized) Support Vector Machine (SVM) training? I am asking this question because I need to ...
1
vote
0answers
134 views

How to choose kernel functions for support vector regression

Are there any good resources regarding how to design kernels for regression problems, specifically time-series regression type of problem. I am finding the choice of a kernel for regression extremely ...
0
votes
0answers
124 views

Does kernel trick correspond to real higher dimensions?

I've read that the SVM kernel tricks corresponds to viewing the data in a higher dimension. I can see that the expansion of the kernel looks like a cross-product of many different terms. However, it ...
7
votes
1answer
399 views

Kernel Ridge Regression Efficiency

Ridge Regression can be expressed as $$\hat{y} = (\mathbf{X'X} + a\mathbf{I}_d)^{-1}\mathbf{X}x$$ where $\hat{y}$ is the predicted label, $\mathbf{I}_d$ the $d \times d$ identify matrix, $\mathbf{x}$ ...
1
vote
1answer
244 views

Proof of closeness of kernel functions under pointwise product

How can I prove that pointwise product of two kernel functions is a kernel function?
9
votes
1answer
861 views

What function could be a kernel?

In the context of machine learning and pattern recognition, there's a concept called Kernel Trick. Facing problems where I am asked to determine whether a function could be a kernel function or not, ...
7
votes
1answer
2k views

What are the advantages of Multiple Kernel Learning (MKL) methods?

Multiple Kernel Learning methods aim to construct a kernel model where the kernel is a linear combination of fixed base kernels. Learning the kernel then consists of learning the weighting ...
0
votes
1answer
102 views

SVM decision function

our decision function e.g. in SVMs for binary classification (where the response is labeld by $y_i \in \{-1,1\}$) has the form: $f(\mathbf{x}) = \text{sgn}(\mathbf{w}^\top \mathbf{x} + b)$ where ...
1
vote
1answer
94 views

Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen?

Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen? If so, to what dimension does it transform the data?
3
votes
1answer
556 views

Kernelised k Nearest Neighbour

I'm new to kernels and have hit a snag while trying to kernelise kNN. Preliminaries I'm using a polynomial kernel: $K(\mathbf{x},\mathbf{y}) = (1 + \langle \mathbf{x},\mathbf{y} \rangle)^d$ Your ...
1
vote
1answer
1k views

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function: $$ ...
0
votes
1answer
336 views

Free data set for comparing kernel logistic regression and regular logistic regression [closed]

I'm looking for a data set that is easily accessible for comparing Kernel Logistic Regression (KLR) and regular logistic regression. All the paper that I find using KLR use synthetic data sets. I'm ...
1
vote
0answers
84 views

Polynomial kernel in logistic regression?

So I have put together a nice logistic regression program that works quite well. Now, I have used two dimensions to test it and see how it works, and guided by some online tutorials, have increased ...
3
votes
0answers
64 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
22
votes
4answers
6k views

How to select kernel for SVM?

When using SVM, we need to select a kernel. I wonder how to select a kernel. Any criteria on kernel selection?