The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
1answer
38 views

Linear Kernel in Baysian Linear Regression

I came up with http://mlg.eng.cam.ac.uk/duvenaud/cookbook/index.html and it is actually very useful. At some point it says If you use just a linear kernel in a GP, you're simply doing Bayesian ...
1
vote
0answers
19 views

Derive squared exponential covariance function

In Gaussian Processes, SVMs, kernels are used (as to my understanding) as similarity measure. However, they have the constraint that any kernel has to be represented as a dot product. i.e. ...
0
votes
0answers
19 views

What's wrong with the intuition that kernel measures similarity between observations?

Near the middle of page 16 of Andrew Ng's notes on SVM, he explained an intuitive view of kernel as measuring similarity between observations, but then added the caveat that there are things wrong ...
0
votes
0answers
16 views

KPCA in R proportion of variability explained

I'm using kpca function from kernlab and try to get the proportion of variance explained by each component as in standard pca. I ...
2
votes
2answers
129 views

The difference of kernels in SVM?

Can someone please tell me the difference between the kernels in SVM: Linear Polynomial Gaussian (RBF) Sigmoid Because as we know that kernel is used to mapped our input space into high ...
2
votes
2answers
52 views

Is this a decent summary of the kernel trick?

Here's my understanding of the kernel trick. The motivation is to find a linear separator in a higher dimensional space than what you have (because the data are not currently linearly separable.) ...
9
votes
3answers
42 views

Should I use the Kernel Trick whenever possible for non-linear data?

I recently learned about the use of the Kernel trick, which maps data into higher dimensional spaces in an attempt to linearize the data in those dimensions. Are there any cases where I should avoid ...
0
votes
0answers
44 views

Explicit mapping of input space to a high dimensional feature space

I am now studying on Kernel Methods, all the theory behind it to understand Support Vector Machines. Of course, I understood some very well but there is something I could not completely comprehend. ...
1
vote
1answer
61 views

Kernel PCA with an SVD algo

Suppose that I have a great algo for calculating the SVD and I want to do Kernel PCA. It is possible to first apply the Kernel function to my data and then run the SVD algo on the transformed data?
3
votes
0answers
43 views

Kernel interpretation of Manhattan distance / Lp norms

Euclidean distance corresponds to the linear kernel similarity via: $$d_\text{Euclidean}(x,y)^2=\sum_i (x_i-y_i)^2=\sum_i x_i^2 + y_i^2 - 2 x_i y_i = ...
0
votes
0answers
29 views

How to project test data onto the principal vectors produced by kpca?

So I have a dataset of dimension 6395x15 and when I apply kernel-PCA on this dataset, I get a rotated matrix of dimesion 6395x596. pcv() gives me the matrix of column-wise eigen-vectors and this ...
0
votes
1answer
95 views

Applying kernel function to input data before giving it to algorithm

I have gene expression data, I do dimensionality reduction and clustering with self organizing maps, but self organizing maps do not perform well with my data. I want to map my data to feature space ...
1
vote
1answer
98 views

Non-Orthogonality in PCA?

i) What is the main role of "only" trying to find orthogonal components in PCA? I can understand, that we would not want a zero-solution as well as find directions that are orthogonal in order to ...
7
votes
1answer
137 views

Kendall-tau and RKHS spaces

Given two random variables $X_1$ and $X_2$, the Kendall-tau correlation coefficient could be defined as $$ ...
0
votes
1answer
41 views

is it possible to use kernel function to calculate each instance of covariance matrix? If Yes why?

I saw a paper that uses Gaussian kernel for calculation at covariance matrix of given variables. Is it mathematically Correct or not? if it is okay,what is the intuition behind ? What about using any ...
1
vote
0answers
23 views

Confusion related to convexity of a function

I was reading this paper http://www.umiacs.umd.edu/~jags/pdfs/KSforNLP.pdf where they say that $trace(K \pi^{T} L \pi)$ is a convex function of $\pi$ where $\pi$ is a permutation matrix and K and L ...
1
vote
0answers
112 views

Generalized RBF Kernels

There is the notion of Generalized RBF Kernels, for example in "Towards Optimal Bag-of-Features for Object Categorization and Semantic Video Retrieval" from Jiang (1) or in formula (2.72) in ...
0
votes
1answer
607 views

Use Gaussian RBF kernel for mapping of 2D data to 3D

I am working on SVMs and try to get all the concepts involved. For instance, the kernel mapping. I would like to construct some parts of the algorithm by myself, to understand what is happening. My ...
1
vote
1answer
156 views

Prove (or disprove) that this function is a kernel

I devised a distance function similar to this form $d(x,y) = \sum_{i = 1}^{n-1} b(x_i, y_i,x_{i+1}, y_{i+1}) $ with $b(x_i, y_i,x_{i+1}, y_{i+1}) = 0 \mbox{ if } x_i \leq 0 \vee y_i \leq 0 \vee ...
1
vote
1answer
457 views

Kernel PCA (in R)

I am attempting to use the kernel PCA features in kernlab but am having trouble understanding the output. In particular, it's unclear what scale the results are in ...
1
vote
1answer
171 views

Number of kernel evaluations in SVM training

What is the typical number of kernel evaluations (between two training vectors) performed during a (kernelized) Support Vector Machine (SVM) training? I am asking this question because I need to ...
1
vote
0answers
118 views

How to choose kernel functions for support vector regression

Are there any good resources regarding how to design kernels for regression problems, specifically time-series regression type of problem. I am finding the choice of a kernel for regression extremely ...
0
votes
0answers
115 views

Does kernel trick correspond to real higher dimensions?

I've read that the SVM kernel tricks corresponds to viewing the data in a higher dimension. I can see that the expansion of the kernel looks like a cross-product of many different terms. However, it ...
7
votes
1answer
347 views

Kernel Ridge Regression Efficiency

Ridge Regression can be expressed as $$\hat{y} = (\mathbf{X'X} + a\mathbf{I}_d)^{-1}\mathbf{X}x$$ where $\hat{y}$ is the predicted label, $\mathbf{I}_d$ the $d \times d$ identify matrix, $\mathbf{x}$ ...
1
vote
1answer
206 views

Proof of closeness of kernel functions under pointwise product

How can I prove that pointwise product of two kernel functions is a kernel function?
8
votes
1answer
680 views

What function could be a kernel?

In the context of machine learning and pattern recognition, there's a concept called Kernel Trick. Facing problems where I am asked to determine whether a function could be a kernel function or not, ...
6
votes
1answer
1k views

What are the advantages of Multiple Kernel Learning (MKL) methods?

Multiple Kernel Learning methods aim to construct a kernel model where the kernel is a linear combination of fixed base kernels. Learning the kernel then consists of learning the weighting ...
0
votes
1answer
96 views

SVM decision function

our decision function e.g. in SVMs for binary classification (where the response is labeld by $y_i \in \{-1,1\}$) has the form: $f(\mathbf{x}) = \text{sgn}(\mathbf{w}^\top \mathbf{x} + b)$ where ...
1
vote
1answer
89 views

Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen?

Is $f(x)=e^{x^Tx'}$ a suitable kernel to be choosen? If so, to what dimension does it transform the data?
2
votes
1answer
459 views

Kernelised k Nearest Neighbour

I'm new to kernels and have hit a snag while trying to kernelise kNN. Preliminaries I'm using a polynomial kernel: $K(\mathbf{x},\mathbf{y}) = (1 + \langle \mathbf{x},\mathbf{y} \rangle)^d$ Your ...
1
vote
1answer
880 views

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function: $$ ...
0
votes
1answer
291 views

Free data set for comparing kernel logistic regression and regular logistic regression [closed]

I'm looking for a data set that is easily accessible for comparing Kernel Logistic Regression (KLR) and regular logistic regression. All the paper that I find using KLR use synthetic data sets. I'm ...
1
vote
0answers
79 views

Polynomial kernel in logistic regression?

So I have put together a nice logistic regression program that works quite well. Now, I have used two dimensions to test it and see how it works, and guided by some online tutorials, have increased ...
3
votes
0answers
61 views

How to learn similarity of typed/attributed graphs?

I have a question for graph machine learning gurus :). For this project I'm working on, I need to be able to learn similarity between typed graphs. By typed I mean that every vertex and every edge of ...
19
votes
4answers
5k views

How to select kernel for SVM?

When using SVM, we need to select a kernel. I wonder how to select a kernel. Any criteria on kernel selection?