Questions tagged [rbf-kernel]

The RBF kernel, i.e., radial-basis-function kernel, occurs in the context of kernel methods in machine learning.

Filter by
Sorted by
Tagged with
3 votes
2 answers
66 views

Distribution of an RBF-transformed normal variable

My question might be related to this or this one, but I have reasons to hope my problem is more benign. Assume I have a normally distributed variable $X \sim N(0, 1)$. What can be said of $Y = \exp(-\...
user avatar
  • 6,363
1 vote
1 answer
14 views

How are centers in an RBF Network chosen?

I am struggling to understand how RBF (radial basis functions) work. My first question concerns the weights: are the learnable weights the same as the centres? So, is the algorithm essentially ...
user avatar
  • 63
0 votes
0 answers
38 views

Implementing kernel alignment for SVM algorithms

I am trying to understand and re-implement the results from Table 2 in the first Kernel-Target Alignment paper. The task that is being done is a simple classification task using an SVM with RBF ...
user avatar
0 votes
0 answers
9 views

The derivation of the RBF kernel to the inner product form, and what does this notation $\sum _{n_{1}+n_{2}+\dots +n_{k}=j}$ mean?

I have two questions regarding the following derivation: What does this notation $\sum _{n_{1}+n_{2}+\dots +n_{k}=j}$ mean in the following equation? How is it derived from step 3 to step 4, and from ...
user avatar
0 votes
0 answers
13 views

What's the effect of Heteroscedasticity in predictor variables for SVM classification?

It's usually a good practice as part of the modeling stage to apply transformations to all predictor variables so that they are stationary or in other words, that the statistical properties such as ...
user avatar
  • 895
0 votes
0 answers
32 views

How much do the hyperparameters matter to a Gaussian Process?

While a Gaussian Process is completely defined by its mean and covariance functions, how much do the hyperparameters of the covariance function matter to the GP? Take the RBF covariance function for ...
user avatar
0 votes
0 answers
50 views

Why are radial basis functions so different from classic inner product?

I was studying SVM with kernel tricks and it seems that the kernel is a modified dot product. A simple kernel would be $K(x,y) = <x,y>^2$. I understand how this is a modification of the dot ...
user avatar
  • 1
0 votes
1 answer
126 views

Why do my GaussianProcessRegressor prediction results converge to 0?

I am using sklearn GaussianProcessRegressor to predict a time series. The kernel I use is this: ...
user avatar
1 vote
0 answers
23 views

Explanation of the multiplier of gaussian process kernels in sklearn document

I have read the basic materials about gaussian process regression and understand its ideas. https://scikit-learn.org/stable/modules/gaussian_process.html However, when I look into the sklearn page, I ...
user avatar
0 votes
0 answers
33 views

Gaussian Process Covariance Guaranteed to be PSD?

I have a question regarding a proof to show that the covariance matrix of a Gaussian process is Positive SemiDefinite (PSD). Given the equation, $cov(\bar{f}) = K_{**} - K_{*f}K_{ff}^{-1}K_{f*}$ how ...
user avatar
0 votes
0 answers
39 views

Does a gaussian kernel suffer from the curse of dimensionality?

Some embedding methods map a data vector in original space to a new space with significantly high dimension and then calculate dot product between these mapped high dimensional vectors. Don't they ...
user avatar
0 votes
1 answer
54 views

Can different kernels be used when performing Gaussian Process Regression?

Given the equations for exact Gaussian process regression: \begin{equation} \bar{\boldsymbol{f}_*} = \boldsymbol{m}(X_*) + K_{*f}(K_{ff} + \sigma^2I_N)^{-1}K_{f*}(\boldsymbol{y} - \boldsymbol{m}(X)), \...
user avatar
0 votes
0 answers
30 views

Kernel trick in feature space

I am working on KPCA based fault detection. I have question concerning the kernel trick in the feature space. We all know that the dot product in the feature space is computed using kernel function. I ...
user avatar
0 votes
0 answers
21 views

Featue map for RBF kenel when dimension is more than 1

i saw the post that write the feature map for RBF kernel but that was when dimension was 1 can anybody help me writing feature map for higher dimensions?
user avatar
0 votes
0 answers
38 views

How do I interpolate a field that is divergence-free and curl-free at the same time?

A magnetic field is divergence free. At the points where there is no current, and no changing electric field, it is also curl free. There exist divergence-free and curl-free RBF kernels, and I could ...
user avatar
  • 101
0 votes
0 answers
20 views

Output Distribution of RBF Kernel

Suppose we know the distirbution of the two variables $x\sim \mathcal{N}(\mu_x, \Sigma_x)$, $y\sim \mathcal{N}(\mu_y, \Sigma_y)$. What does the resulting distribution of $k(x,y)=\exp(-\frac{|| x-y||^...
user avatar
0 votes
1 answer
145 views

Is it efficient to use kernel trick in primal form of SVM?

I know we can use Kernel trick in the primal form of SVM. So the hypothesis will be - and optimization objective - We can optimize the above equation using gradient descent, but in this equation ...
user avatar
3 votes
0 answers
108 views

How to obtain the inverse of the Gram (kernel) matrix?

We're working with a similar dual SVR problem that involves the inversion of a Gram (kernel) matrix: $\boldsymbol{S}_{i,j} = e^{ -\gamma ||\vec{x_i} - \vec{x_j}||_2^2}$ With some data-sets (e.g.: UCI ...
user avatar
1 vote
0 answers
194 views

Can someone clarify what the linear assumption of PCA is?

For the past few hours I've been trying to search what this linear assumption is. Some of the articles states that that your independent variables have to be linear in relationship and need some type ...
user avatar
5 votes
2 answers
195 views

Why use RBF kernel if less is needed?

I have seen online theorem's such as Cover's theorem Wikipedia which prove how given $p$ points in $\mathbb{R}^N$ the linear separability is almost certain as the fraction $\dfrac{p}{N}$ is kept close ...
user avatar
5 votes
1 answer
112 views

Using Gaussian Processes to learn a function online

I would like to approximate a function $f:\mathbb{R} \to \mathbb{R}_+$ based on a set of samples. I obtain these samples online (i.e. sequentially in time). That is, at time $t$ I receive $(x_t, f(x_t)...
user avatar
1 vote
1 answer
232 views

What is the difference between a covariance matrix created by an RBF kernel and a covariance matrix created by

I can't explain something simple to myself and it is probably a matter of vocabulary, I am not sure... If I create and random normal $Z \in \mathbb{R}^{3\times5}$, each row and column has a mean of 0. ...
user avatar
  • 629
3 votes
0 answers
32 views

In the broadest sense, what is a "kernel"?

In MCMC sampling methods, a transition kernel, as found in Metropolis(/Hastings) algorithm, is the comparison of the likelihood of the current position and the likelihood of the proposed position. ...
user avatar
  • 1,634
2 votes
1 answer
96 views

Prove that the following matrix is positive definite

We define $K_{\mathbf{a}, \mathbf{b}}$ as the $n \times m$ matrix whose $ij^{th}$ entry is $\kappa(a_{i}, b_{j})$ Where, $\kappa$ is a (positive definite) kernel function. Here, $\mathbf{a}_{i}, \...
user avatar
0 votes
1 answer
287 views

About Gaussian kernel for distances other than Euclidian

I have a question about Gaussian kernel. I read the following site. https://datascience.stackexchange.com/questions/17352/why-do-we-use-a-gaussian-kernel-as-a-similarity-metric My question is whether ...
user avatar
1 vote
0 answers
296 views

How to use sklearn's Gaussian Process Regression parameters?

I have been trying to play around with Gaussian process Regression. I have constructed a fake 1D data for this. I am using a Squared exponential kernel. I solved the regression problem using inbuilt ...
user avatar
1 vote
0 answers
47 views

How to extend kernel-based classifier to non-euclidean space like SO3

What is the proper way to extend kernel-based classifier to non-euclidean space like SO3? This kind of situation happens a lot in robotics, where the data points all live in a specific manifold. (Note:...
user avatar
1 vote
0 answers
31 views

Can someone improve my feeble understanding of feature maps and kernels?

I am taking the course and the extent in which we've discussed feature maps and kernels is as follows: Given Obviously we cannot use linear regression. Instead we map it to a space where it becomes ...
user avatar
0 votes
1 answer
150 views

Does radial basis function kernel has a coefficient?

I found there are two forms of RBF function. these is a coefficient before $\exp$ $$ k_{f}\left(x_{i}, x_{j}\right)=\sigma^{2} \exp \left(-\frac{1}{2 \ell^{2}} \...
user avatar
1 vote
1 answer
214 views

Calculation of Lipschiz constant for Square Exponential kernel

I am working with the Kernel function and want to calculate bounds using the concept of Lipschitz continuity. I do understand that the SE kernel is continuous, smooth, and differentiable. Is there any ...
user avatar
3 votes
0 answers
2k views

Why SVM with gamma='scale' for RBF kernel works so well?

The intuitive explanation for the gamma parameter of the RBF kernel in SVMs is the following: Intuitively, the gamma parameter ...
user avatar
1 vote
1 answer
653 views

Is it possible to find cluster centroids in kernel K means?

Suppose ${x_1, \ldots, x_N}$ are the data points and we have to find $K$ clusters using Kernel K Means. Let the kernel be $Ker$ (not to confuse with $K$ number of clusters) Let $\phi$ be the implicit ...
user avatar
  • 159
1 vote
0 answers
97 views

Are radial basis kernels able to model interactions between predictors?

I have been doing research using Support Vector Regression for some time, especially using radial basis kernel, for predicting a response variable from a set of numeric predictors. As a consequence of ...
user avatar
1 vote
1 answer
138 views

SVR with combination of kernels

I am a beginner, and I am looking for some advice regarding the use of Support Vector Regression (SVR) to model (or fit if you prefer) a trend. Before you suggest other methods, for a number of ...
user avatar
  • 11
1 vote
2 answers
70 views

Relationship between structural or statistical properties and hardness of classification

I am trying to understand the relationship between structural or statistical properties of training dataset and hardness of classification in the context of binary classification with SVM using RBF ...
user avatar
1 vote
1 answer
125 views

What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $k$ over any space $\mathcal{X}$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $k$. For example, Radial basis ...
user avatar
1 vote
0 answers
103 views

How is a polynomial kernel with infinite degree different from RBF Kernel?

I was reading about polynomial and RBF Kernels. According to my understanding: Polynomial kernels with degree >1 map the non-linear data into a higher dimensional feature space. Data that aren't ...
user avatar
2 votes
1 answer
1k views

Kernel approximation with Nystroem method and usage in scikit-learn

I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn. As far as I understood, the full Gram Matrix ...
user avatar
0 votes
0 answers
15 views

kernel and mean function for series of function

my series for functions are type. $$f(x) = a \sin(x-b) , a \sim \mathcal{N}(-1,2) , b \sim \mathcal{N}(-0.5,1)$$ Can someone get me started how to model these functions with GP. I am confused about ...
user avatar
1 vote
1 answer
123 views

Finding optimal kernel parameters

I want to perform multiple kernel learning on my dataset and apply each (rbf) kernel to a different subset of features to then combine them. I do not want to have the same kernel with a range of ...
user avatar
  • 85
0 votes
0 answers
20 views

Applying different kernels to parts of a dataset and merging the output [duplicate]

I am trying to create a classifier using SVM on a dataset that is composed of 6 sets of data for each of my observations. When I train the SVM (rbf kernel), I get a better performance of the ...
user avatar
  • 85
0 votes
0 answers
845 views

Calculate Gamma for RBF Kernel to get Gaussian Kernel

In order to measure the information density like proposed in section 3.2 of this paper I need a symmetric positive definite Kernel function. For this purpose I want to use the Gaussian Kernel like ...
user avatar
1 vote
2 answers
491 views

hyperparameters optimisation with linear kernel

I want to conduct an SVM model-regression (i.e., support vector regression), using a linear kernel function. Does it make sense to perform a cross-validation hyperparameter optimization when the ...
user avatar
  • 41
0 votes
1 answer
316 views

Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
user avatar
1 vote
0 answers
159 views

What is the role of length scale bound in sklearn Radial Basis Function

The radial basis function provided by SkLearn (reference) has two parameters: length scale and length scale bounds. I understand that the length scale controls the importance of the coordinates of the ...
user avatar
2 votes
1 answer
200 views

What is the expanded representation, $\phi(X)$, required to obtain the RBF kernel?

For the two-dimensional case, where $\boldsymbol X=[x_1, x_2]$ and its corresponding expanded represetation $\boldsymbol\phi(X)= [1, \sqrt2 x_1, \sqrt2 x_2, x_1^2, x_2^2, \sqrt2x_1x_2]$, we can ...
user avatar
  • 531
0 votes
1 answer
58 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
user avatar
  • 47
1 vote
0 answers
22 views

Is it good idea to generate features from data points similarity comparison?

I know about polynomial features in machine learning, which can introduce nonlinearity to original dataset. I also heard about binning, which also allows us to create new features from existing ones. ...
user avatar
  • 291
13 votes
1 answer
3k views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
user avatar
  • 1,432
1 vote
1 answer
43 views

Is SVM RBF applied to both classes?

Lets say i have following 1D data (position on x), color is target class and I need a classifier which classifies green from red: I decided to use SVM. Data is clearly not linearly separable, so i ...
user avatar
  • 291