Questions tagged [rbf-kernel]

The RBF kernel, i.e., radial-basis-function kernel, occurs in the context of kernel methods in machine learning.

Filter by
Sorted by
Tagged with
3
votes
1answer
34 views

Using Gaussian Processes to learn a function online

I would like to approximate a function $f:\mathbb{R} \to \mathbb{R}_+$ based on a set of samples. I obtain these samples online (i.e. sequentially in time). That is, at time $t$ I receive $(x_t, f(x_t)...
1
vote
1answer
17 views

What is the difference between a covariance matrix created by an RBF kernel and a covariance matrix created by

I can't explain something simple to myself and it is probably a matter of vocabulary, I am not sure... If I create and random normal $Z \in \mathbb{R}^{3\times5}$, each row and column has a mean of 0. ...
2
votes
0answers
23 views

In the broadest sense, what is a “kernel”?

In MCMC sampling methods, a transition kernel, as found in Metropolis(/Hastings) algorithm, is the comparison of the likelihood of the current position and the likelihood of the proposed position. ...
3
votes
1answer
630 views

Calculation of nu and gamma in one-class SVM with rbf kernel

I am using python sklearn's one-class svm classifier for anomaly detection. I would like to know can I accurately calculate the required value for nu and gamma for rbf kernel. Is there any equation or ...
0
votes
0answers
13 views

RBF networks and epsilon-NN

Will an RBF network with a Window kernel correspond to an epsilon-NN classifier? I know that kNN only considers the k nearest data points while RBF uses all the data. How does this relate to epsilon-...
2
votes
1answer
29 views

Prove that the following matrix is positive definite

We define $K_{\mathbf{a}, \mathbf{b}}$ as the $n \times m$ matrix whose $ij^{th}$ entry is $\kappa(a_{i}, b_{j})$ Where, $\kappa$ is a (positive definite) kernel function. Here, $\mathbf{a}_{i}, \...
0
votes
1answer
35 views

About Gaussian kernel for distances other than Euclidian

I have a question about Gaussian kernel. I read the following site. https://datascience.stackexchange.com/questions/17352/why-do-we-use-a-gaussian-kernel-as-a-similarity-metric My question is whether ...
0
votes
0answers
21 views

What are my options for fitting a model with unbounded moments

I'm trying to fit a model on a distribution that likely has unbounded moments. Say it's some stable distribution generated from a linear combination of random shocks with unbounded at least variance, ...
1
vote
0answers
37 views

How to use sklearn's Gaussian Process Regression parameters?

I have been trying to play around with Gaussian process Regression. I have constructed a fake 1D data for this. I am using a Squared exponential kernel. I solved the regression problem using inbuilt ...
1
vote
0answers
19 views

How to extend kernel-based classifier to non-euclidean space like SO3

What is the proper way to extend kernel-based classifier to non-euclidean space like SO3? This kind of situation happens a lot in robotics, where the data points all live in a specific manifold. (Note:...
1
vote
0answers
19 views

Can someone improve my feeble understanding of feature maps and kernels?

I am taking the course and the extent in which we've discussed feature maps and kernels is as follows: Given Obviously we cannot use linear regression. Instead we map it to a space where it becomes ...
0
votes
1answer
26 views

Does radial basis function kernel has a coefficient?

I found there are two forms of RBF function. these is a coefficient before $\exp$ $$ k_{f}\left(x_{i}, x_{j}\right)=\sigma^{2} \exp \left(-\frac{1}{2 \ell^{2}} \...
0
votes
0answers
26 views

Kernel PCA stopping rule / criterion

I have been comparing dimensionality reduction methods, while performing KPCA, I didn't know what number of PCs I should retain, how many to take and decide : "there it is, this is the new ...
0
votes
0answers
11 views

Trying to use a custom kernel in SVR

I am very new to python and ML, I'm trying to customize a RBF kernel to consider Mahalanobis distance instead of Euclidean distance but I am encountering problems when I try to predict on new ...
1
vote
1answer
42 views

Calculation of Lipschiz constant for Square Exponential kernel

I am working with the Kernel function and want to calculate bounds using the concept of Lipschitz continuity. I do understand that the SE kernel is continuous, smooth, and differentiable. Is there any ...
0
votes
2answers
326 views

SVM and correlation

Can anyone guide me about the feature selection based on correlation using SVM? RBF kernel check the correlation too or not? I am using weka and matlab. Any help would be appreciated.
2
votes
0answers
274 views

Why SVM with gamma='scale' for RBF kernel works so well?

The intuitive explanation for the gamma parameter of the RBF kernel in SVMs is the following: Intuitively, the gamma parameter ...
0
votes
0answers
14 views

Use cases exist for the Silverman kernel

I am well aware of the usage of the Gaussian, the boxcar function, the Epanechnikov function and others for use cases as kernel density estimation, Gaussian processes and others. But I have never seen ...
1
vote
1answer
124 views

Is it possible to find cluster centroids in kernel K means?

Suppose ${x_1, \ldots, x_N}$ are the data points and we have to find $K$ clusters using Kernel K Means. Let the kernel be $Ker$ (not to confuse with $K$ number of clusters) Let $\phi$ be the implicit ...
1
vote
0answers
28 views

Are radial basis kernels able to model interactions between predictors?

I have been doing research using Support Vector Regression for some time, especially using radial basis kernel, for predicting a response variable from a set of numeric predictors. As a consequence of ...
1
vote
1answer
399 views

Kernel approximation with Nystroem method and usage in scikit-learn

I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn. As far as I understood, the full Gram Matrix ...
1
vote
1answer
49 views

SVR with combination of kernels

I am a beginner, and I am looking for some advice regarding the use of Support Vector Regression (SVR) to model (or fit if you prefer) a trend. Before you suggest other methods, for a number of ...
0
votes
2answers
46 views

Relationship between structural or statistical properties and hardness of classification

I am trying to understand the relationship between structural or statistical properties of training dataset and hardness of classification in the context of binary classification with SVM using RBF ...
1
vote
1answer
43 views

What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $k$ over any space $\mathcal{X}$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $k$. For example, Radial basis ...
1
vote
0answers
62 views

How is a polynomial kernel with infinite degree different from RBF Kernel?

I was reading about polynomial and RBF Kernels. According to my understanding: Polynomial kernels with degree >1 map the non-linear data into a higher dimensional feature space. Data that aren't ...
0
votes
0answers
23 views

heuristics for gamma in rbf kernel

My question is a follow-up to this question: SVM rbf kernel - heuristic method for estimating gamma. Basically, I want to find interesting values for gamma by first calculating the pairwise distance ...
0
votes
0answers
18 views

Can I customize the kernel function?

I want to know whether I can customize the kernel function? For example, the polynomial kernel is defined as: $$ K(x,y) = (x^Ty+c)^d $$ Could I modify it to the following: $$ K(x,y) = (||x-y||_2)^d $$ ...
0
votes
1answer
43 views

Finding optimal kernel parameters

I want to perform multiple kernel learning on my dataset and apply each (rbf) kernel to a different subset of features to then combine them. I do not want to have the same kernel with a range of ...
0
votes
0answers
13 views

kernel and mean function for series of function

my series for functions are type. $$f(x) = a \sin(x-b) , a \sim \mathcal{N}(-1,2) , b \sim \mathcal{N}(-0.5,1)$$ Can someone get me started how to model these functions with GP. I am confused about ...
0
votes
0answers
16 views

Applying different kernels to parts of a dataset and merging the output [duplicate]

I am trying to create a classifier using SVM on a dataset that is composed of 6 sets of data for each of my observations. When I train the SVM (rbf kernel), I get a better performance of the ...
0
votes
0answers
68 views

Using Gaussian Process Regression in scikit-learn

I have a simple dataset with multiple trials of position over time, and I'm trying to fit a Gaussian Process over it. Here's a plot of all the raw data (6180 data points): My goal is to fit a ...
1
vote
0answers
285 views

Heavy-tailed RBF kernel function

I'm trying to run a SVM regression on some data and I want to use ksvm from kernlab or svm ...
0
votes
0answers
35 views

Using RBF k-NN graph in spectral clustering

In the article Spectral Clustering with Imbalanced Data there is mentioned usage of a "RBF k-NN" graph. I haven't encountered this kind of graphs before and couldn't google anything related to it. ...
0
votes
0answers
450 views

Calculate Gamma for RBF Kernel to get Gaussian Kernel

In order to measure the information density like proposed in section 3.2 of this paper I need a symmetric positive definite Kernel function. For this purpose I want to use the Gaussian Kernel like ...
1
vote
2answers
179 views

hyperparameters optimisation with linear kernel

I want to conduct an SVM model-regression (i.e., support vector regression), using a linear kernel function. Does it make sense to perform a cross-validation hyperparameter optimization when the ...
0
votes
1answer
104 views

Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
1
vote
0answers
108 views

What is the role of length scale bound in sklearn Radial Basis Function

The radial basis function provided by SkLearn (reference) has two parameters: length scale and length scale bounds. I understand that the length scale controls the importance of the coordinates of the ...
1
vote
1answer
63 views

What is the expanded representation, $\phi(X)$, required to obtain the RBF kernel?

For the two-dimensional case, where $\boldsymbol X=[x_1, x_2]$ and its corresponding expanded represetation $\boldsymbol\phi(X)= [1, \sqrt2 x_1, \sqrt2 x_2, x_1^2, x_2^2, \sqrt2x_1x_2]$, we can ...
0
votes
1answer
35 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
12
votes
1answer
1k views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
1
vote
0answers
20 views

Is it good idea to generate features from data points similarity comparison?

I know about polynomial features in machine learning, which can introduce nonlinearity to original dataset. I also heard about binning, which also allows us to create new features from existing ones. ...
1
vote
1answer
35 views

Is SVM RBF applied to both classes?

Lets say i have following 1D data (position on x), color is target class and I need a classifier which classifies green from red: I decided to use SVM. Data is clearly not linearly separable, so i ...
1
vote
0answers
236 views

Gaussian RBF vs KNN explanation

I was studying SVM ML alghorythm and I was wondering about solution for non-linear cases. As I understand it for know, SVM tries to find hyperplane or object in defined n-dimensional space, which ...
1
vote
0answers
24 views

Labeling KPCA in R

The function $\texttt{biplot}$ in R is very useful for creating visualizations when performing PCA. However, when performing kernel PCA in R, I cannot find a way to label the loadings on the graph. ...
1
vote
1answer
142 views

Gaussian processes Sum of RBF kernels vs single anisotropic RBF kernel [closed]

Say I have some two dimensional data, for which I am trying to fit a Gaussian process. In scikit-learn, I can build an RBF kernel as follows K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + ...
3
votes
1answer
66 views

The inner product properties seem to clash with the RKHS property for RBF kernels. What is off?

By the reproducing kernel Hilbert space (RKHS) property, given a P.S.D. kernel function $\kappa:X\times X \rightarrow \mathbb R$, there exists a Hilbert space $H$ and a map $\phi:X\rightarrow H$ such ...
0
votes
2answers
334 views

RBF kernel mapping

I was reading that the Gaussian/RBF kernel maps its input onto the surface of normalized hypersphere. Our RBF kernel given by: $k(x,z) = exp(\frac{- ||x-z||^2}{2\sigma^2})$ Can anyone explain why ...
2
votes
0answers
340 views

Kernel function with a feature space equipped with an inner product that is not the dot product

Premise: A function $K: \mathbb R^d \times \mathbb R^d \to \mathbb R$ is called a kernel function on $\mathbb{R}^d$ if there exists a Hilbert space $\mathcal{H}$ and a map $\phi: \mathbb R^d \...
1
vote
1answer
104 views

Fundamental understanding of Gaussian Process and their terminology [closed]

I am new to this site as well as Machine learning, so kindly bear with me. I have been trying to understand Gaussian process and their implementation. Notation: 1) Let's say that the $\vec{x}$ $\in ...
1
vote
0answers
317 views

What is the most intuitive proof that Gaussian kernel is positive definite? [duplicate]

I have general form of Gaussian kernel $K(x,x')=\exp(-\|x-x'\|^{2})$ (just not considering $\sigma$). I tried to prove its positive definiteness via Gram matrix properties, but couldn't. Is there any ...