Questions tagged [rbf-kernel]

The RBF kernel, i.e., radial-basis-function kernel, occurs in the context of kernel methods in machine learning.

Filter by
Sorted by
Tagged with
0
votes
1answer
15 views

Is it efficient to use kernel trick in primal form of SVM?

I know we can use Kernel trick in the primal form of SVM. So the hypothesis will be - and optimization objective - We can optimize the above equation using gradient descent, but in this equation ...
0
votes
0answers
25 views

How to obtain the inverse of the Gram (kernel) matrix? [closed]

We're working with a similar dual SVR problem that involves the inversion of a Gram (kernel) matrix: $\boldsymbol{S}_{i,j} = e^{ -\gamma ||\vec{x_i} - \vec{x_j}||_2^2}$ With some data-sets (e.g.: UCI ...
1
vote
0answers
42 views

Can someone clarify what the linear assumption of PCA is?

For the past few hours I've been trying to search what this linear assumption is. Some of the articles states that that your independent variables have to be linear in relationship and need some type ...
4
votes
1answer
61 views

Why use RBF kernel if less is needed?

I have seen online theorem's such as Cover's theorem Wikipedia which prove how given $p$ points in $\mathbb{R}^N$ the linear separability is almost certain as the fraction $\dfrac{p}{N}$ is kept close ...
4
votes
1answer
96 views

Using Gaussian Processes to learn a function online

I would like to approximate a function $f:\mathbb{R} \to \mathbb{R}_+$ based on a set of samples. I obtain these samples online (i.e. sequentially in time). That is, at time $t$ I receive $(x_t, f(x_t)...
1
vote
1answer
34 views

What is the difference between a covariance matrix created by an RBF kernel and a covariance matrix created by

I can't explain something simple to myself and it is probably a matter of vocabulary, I am not sure... If I create and random normal $Z \in \mathbb{R}^{3\times5}$, each row and column has a mean of 0. ...
2
votes
0answers
23 views

In the broadest sense, what is a “kernel”?

In MCMC sampling methods, a transition kernel, as found in Metropolis(/Hastings) algorithm, is the comparison of the likelihood of the current position and the likelihood of the proposed position. ...
0
votes
0answers
16 views

RBF networks and epsilon-NN

Will an RBF network with a Window kernel correspond to an epsilon-NN classifier? I know that kNN only considers the k nearest data points while RBF uses all the data. How does this relate to epsilon-...
2
votes
1answer
39 views

Prove that the following matrix is positive definite

We define $K_{\mathbf{a}, \mathbf{b}}$ as the $n \times m$ matrix whose $ij^{th}$ entry is $\kappa(a_{i}, b_{j})$ Where, $\kappa$ is a (positive definite) kernel function. Here, $\mathbf{a}_{i}, \...
0
votes
0answers
24 views

What are my options for fitting a model with unbounded moments

I'm trying to fit a model on a distribution that likely has unbounded moments. Say it's some stable distribution generated from a linear combination of random shocks with unbounded at least variance, ...
0
votes
1answer
39 views

About Gaussian kernel for distances other than Euclidian

I have a question about Gaussian kernel. I read the following site. https://datascience.stackexchange.com/questions/17352/why-do-we-use-a-gaussian-kernel-as-a-similarity-metric My question is whether ...
1
vote
0answers
67 views

How to use sklearn's Gaussian Process Regression parameters?

I have been trying to play around with Gaussian process Regression. I have constructed a fake 1D data for this. I am using a Squared exponential kernel. I solved the regression problem using inbuilt ...
1
vote
0answers
23 views

How to extend kernel-based classifier to non-euclidean space like SO3

What is the proper way to extend kernel-based classifier to non-euclidean space like SO3? This kind of situation happens a lot in robotics, where the data points all live in a specific manifold. (Note:...
1
vote
0answers
20 views

Can someone improve my feeble understanding of feature maps and kernels?

I am taking the course and the extent in which we've discussed feature maps and kernels is as follows: Given Obviously we cannot use linear regression. Instead we map it to a space where it becomes ...
0
votes
1answer
44 views

Does radial basis function kernel has a coefficient?

I found there are two forms of RBF function. these is a coefficient before $\exp$ $$ k_{f}\left(x_{i}, x_{j}\right)=\sigma^{2} \exp \left(-\frac{1}{2 \ell^{2}} \...
0
votes
0answers
34 views

Kernel PCA stopping rule / criterion

I have been comparing dimensionality reduction methods, while performing KPCA, I didn't know what number of PCs I should retain, how many to take and decide : "there it is, this is the new ...
0
votes
0answers
14 views

Trying to use a custom kernel in SVR

I am very new to python and ML, I'm trying to customize a RBF kernel to consider Mahalanobis distance instead of Euclidean distance but I am encountering problems when I try to predict on new ...
1
vote
1answer
54 views

Calculation of Lipschiz constant for Square Exponential kernel

I am working with the Kernel function and want to calculate bounds using the concept of Lipschitz continuity. I do understand that the SE kernel is continuous, smooth, and differentiable. Is there any ...
2
votes
0answers
498 views

Why SVM with gamma='scale' for RBF kernel works so well?

The intuitive explanation for the gamma parameter of the RBF kernel in SVMs is the following: Intuitively, the gamma parameter ...
0
votes
0answers
15 views

Use cases exist for the Silverman kernel

I am well aware of the usage of the Gaussian, the boxcar function, the Epanechnikov function and others for use cases as kernel density estimation, Gaussian processes and others. But I have never seen ...
1
vote
1answer
190 views

Is it possible to find cluster centroids in kernel K means?

Suppose ${x_1, \ldots, x_N}$ are the data points and we have to find $K$ clusters using Kernel K Means. Let the kernel be $Ker$ (not to confuse with $K$ number of clusters) Let $\phi$ be the implicit ...
1
vote
0answers
36 views

Are radial basis kernels able to model interactions between predictors?

I have been doing research using Support Vector Regression for some time, especially using radial basis kernel, for predicting a response variable from a set of numeric predictors. As a consequence of ...
1
vote
1answer
62 views

SVR with combination of kernels

I am a beginner, and I am looking for some advice regarding the use of Support Vector Regression (SVR) to model (or fit if you prefer) a trend. Before you suggest other methods, for a number of ...
0
votes
2answers
46 views

Relationship between structural or statistical properties and hardness of classification

I am trying to understand the relationship between structural or statistical properties of training dataset and hardness of classification in the context of binary classification with SVM using RBF ...
1
vote
1answer
61 views

What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $k$ over any space $\mathcal{X}$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $k$. For example, Radial basis ...
1
vote
0answers
68 views

How is a polynomial kernel with infinite degree different from RBF Kernel?

I was reading about polynomial and RBF Kernels. According to my understanding: Polynomial kernels with degree >1 map the non-linear data into a higher dimensional feature space. Data that aren't ...
0
votes
0answers
32 views

heuristics for gamma in rbf kernel

My question is a follow-up to this question: SVM rbf kernel - heuristic method for estimating gamma. Basically, I want to find interesting values for gamma by first calculating the pairwise distance ...
0
votes
0answers
19 views

Can I customize the kernel function?

I want to know whether I can customize the kernel function? For example, the polynomial kernel is defined as: $$ K(x,y) = (x^Ty+c)^d $$ Could I modify it to the following: $$ K(x,y) = (||x-y||_2)^d $$ ...
1
vote
1answer
491 views

Kernel approximation with Nystroem method and usage in scikit-learn

I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn. As far as I understood, the full Gram Matrix ...
0
votes
0answers
13 views

kernel and mean function for series of function

my series for functions are type. $$f(x) = a \sin(x-b) , a \sim \mathcal{N}(-1,2) , b \sim \mathcal{N}(-0.5,1)$$ Can someone get me started how to model these functions with GP. I am confused about ...
0
votes
1answer
56 views

Finding optimal kernel parameters

I want to perform multiple kernel learning on my dataset and apply each (rbf) kernel to a different subset of features to then combine them. I do not want to have the same kernel with a range of ...
0
votes
0answers
16 views

Applying different kernels to parts of a dataset and merging the output [duplicate]

I am trying to create a classifier using SVM on a dataset that is composed of 6 sets of data for each of my observations. When I train the SVM (rbf kernel), I get a better performance of the ...
0
votes
0answers
510 views

Calculate Gamma for RBF Kernel to get Gaussian Kernel

In order to measure the information density like proposed in section 3.2 of this paper I need a symmetric positive definite Kernel function. For this purpose I want to use the Gaussian Kernel like ...
1
vote
2answers
218 views

hyperparameters optimisation with linear kernel

I want to conduct an SVM model-regression (i.e., support vector regression), using a linear kernel function. Does it make sense to perform a cross-validation hyperparameter optimization when the ...
0
votes
1answer
128 views

Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
1
vote
0answers
117 views

What is the role of length scale bound in sklearn Radial Basis Function

The radial basis function provided by SkLearn (reference) has two parameters: length scale and length scale bounds. I understand that the length scale controls the importance of the coordinates of the ...
1
vote
1answer
67 views

What is the expanded representation, $\phi(X)$, required to obtain the RBF kernel?

For the two-dimensional case, where $\boldsymbol X=[x_1, x_2]$ and its corresponding expanded represetation $\boldsymbol\phi(X)= [1, \sqrt2 x_1, \sqrt2 x_2, x_1^2, x_2^2, \sqrt2x_1x_2]$, we can ...
0
votes
1answer
38 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
1
vote
0answers
20 views

Is it good idea to generate features from data points similarity comparison?

I know about polynomial features in machine learning, which can introduce nonlinearity to original dataset. I also heard about binning, which also allows us to create new features from existing ones. ...
12
votes
1answer
2k views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
1
vote
1answer
36 views

Is SVM RBF applied to both classes?

Lets say i have following 1D data (position on x), color is target class and I need a classifier which classifies green from red: I decided to use SVM. Data is clearly not linearly separable, so i ...
1
vote
0answers
271 views

Gaussian RBF vs KNN explanation

I was studying SVM ML alghorythm and I was wondering about solution for non-linear cases. As I understand it for know, SVM tries to find hyperplane or object in defined n-dimensional space, which ...
1
vote
0answers
28 views

Labeling KPCA in R

The function $\texttt{biplot}$ in R is very useful for creating visualizations when performing PCA. However, when performing kernel PCA in R, I cannot find a way to label the loadings on the graph. ...
1
vote
1answer
159 views

Gaussian processes Sum of RBF kernels vs single anisotropic RBF kernel [closed]

Say I have some two dimensional data, for which I am trying to fit a Gaussian process. In scikit-learn, I can build an RBF kernel as follows K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + ...
3
votes
1answer
68 views

The inner product properties seem to clash with the RKHS property for RBF kernels. What is off?

By the reproducing kernel Hilbert space (RKHS) property, given a P.S.D. kernel function $\kappa:X\times X \rightarrow \mathbb R$, there exists a Hilbert space $H$ and a map $\phi:X\rightarrow H$ such ...
1
vote
1answer
126 views

Fundamental understanding of Gaussian Process and their terminology [closed]

I am new to this site as well as Machine learning, so kindly bear with me. I have been trying to understand Gaussian process and their implementation. Notation: 1) Let's say that the $\vec{x}$ $\in ...
0
votes
2answers
368 views

RBF kernel mapping

I was reading that the Gaussian/RBF kernel maps its input onto the surface of normalized hypersphere. Our RBF kernel given by: $k(x,z) = exp(\frac{- ||x-z||^2}{2\sigma^2})$ Can anyone explain why ...
2
votes
0answers
388 views

Kernel function with a feature space equipped with an inner product that is not the dot product

Premise: A function $K: \mathbb R^d \times \mathbb R^d \to \mathbb R$ is called a kernel function on $\mathbb{R}^d$ if there exists a Hilbert space $\mathcal{H}$ and a map $\phi: \mathbb R^d \...
1
vote
0answers
57 views

Possible error in evaluating kernel gradient in scikit-learn's GPR

Perhaps I am missing something very obvious, but in the standard kernels associated with scikit-learn's Gaussian process regression framework, the radial basis function (RBF), $$f = e^{-x^2/2l^2},$$ ...
1
vote
1answer
114 views

Is this a valid Gaussian Process kernel?

$\mathcal{K}\Big( \; (x,y), (x',y') \; \Big) = \sigma_f^2 \exp{ \frac{(x-x')^2}{2l^2 \cdot (y+y')^2} } $, where $l > 0$ The variance associated with each training point (given by a vector) is a ...