Questions tagged [rbf-kernel]

The RBF kernel, i.e., radial-basis-function kernel, occurs in the context of kernel methods in machine learning.

Filter by
Sorted by
Tagged with
0
votes
1answer
32 views

Why do my GaussianProcessRegressor prediction results converge to 0?

I am using sklearn GaussianProcessRegressor to predict a time series. The kernel I use is this: ...
1
vote
0answers
15 views

Explanation of the multiplier of gaussian process kernels in sklearn document

I have read the basic materials about gaussian process regression and understand its ideas. https://scikit-learn.org/stable/modules/gaussian_process.html However, when I look into the sklearn page, I ...
0
votes
0answers
21 views

Gaussian Process Covariance Guaranteed to be PSD?

I have a question regarding a proof to show that the covariance matrix of a Gaussian process is Positive SemiDefinite (PSD). Given the equation, $cov(\bar{f}) = K_{**} - K_{*f}K_{ff}^{-1}K_{f*}$ how ...
0
votes
0answers
15 views

Does a gaussian kernel suffer from the curse of dimensionality?

Some embedding methods map a data vector in original space to a new space with significantly high dimension and then calculate dot product between these mapped high dimensional vectors. Don't they ...
0
votes
1answer
27 views

Can different kernels be used when performing Gaussian Process Regression?

Given the equations for exact Gaussian process regression: \begin{equation} \bar{\boldsymbol{f}_*} = \boldsymbol{m}(X_*) + K_{*f}(K_{ff} + \sigma^2I_N)^{-1}K_{f*}(\boldsymbol{y} - \boldsymbol{m}(X)), \...
1
vote
0answers
24 views

Kernel trick in feature space

I am working on KPCA based fault detection. I have question concerning the kernel trick in the feature space. We all know that the dot product in the feature space is computed using kernel function. I ...
0
votes
0answers
20 views

Featue map for RBF kenel when dimension is more than 1

i saw the post that write the feature map for RBF kernel but that was when dimension was 1 can anybody help me writing feature map for higher dimensions?
0
votes
0answers
21 views

How do I interpolate a field that is divergence-free and curl-free at the same time?

A magnetic field is divergence free. At the points where there is no current, and no changing electric field, it is also curl free. There exist divergence-free and curl-free RBF kernels, and I could ...
0
votes
0answers
18 views

Output Distribution of RBF Kernel

Suppose we know the distirbution of the two variables $x\sim \mathcal{N}(\mu_x, \Sigma_x)$, $y\sim \mathcal{N}(\mu_y, \Sigma_y)$. What does the resulting distribution of $k(x,y)=\exp(-\frac{|| x-y||^...
0
votes
0answers
17 views

oneClass SVM with rbf kernel

I'm building a model using OneClass SVM. when I use those parameters- OneClassSVM(kernel='linear', gamma='scale', nu=0.01), the 10-fold cross-validation results are around 80%, but the model is not ...
0
votes
1answer
60 views

Is it efficient to use kernel trick in primal form of SVM?

I know we can use Kernel trick in the primal form of SVM. So the hypothesis will be - and optimization objective - We can optimize the above equation using gradient descent, but in this equation ...
3
votes
0answers
56 views

How to obtain the inverse of the Gram (kernel) matrix?

We're working with a similar dual SVR problem that involves the inversion of a Gram (kernel) matrix: $\boldsymbol{S}_{i,j} = e^{ -\gamma ||\vec{x_i} - \vec{x_j}||_2^2}$ With some data-sets (e.g.: UCI ...
1
vote
0answers
100 views

Can someone clarify what the linear assumption of PCA is?

For the past few hours I've been trying to search what this linear assumption is. Some of the articles states that that your independent variables have to be linear in relationship and need some type ...
4
votes
1answer
89 views

Why use RBF kernel if less is needed?

I have seen online theorem's such as Cover's theorem Wikipedia which prove how given $p$ points in $\mathbb{R}^N$ the linear separability is almost certain as the fraction $\dfrac{p}{N}$ is kept close ...
5
votes
1answer
105 views

Using Gaussian Processes to learn a function online

I would like to approximate a function $f:\mathbb{R} \to \mathbb{R}_+$ based on a set of samples. I obtain these samples online (i.e. sequentially in time). That is, at time $t$ I receive $(x_t, f(x_t)...
2
votes
1answer
112 views

What is the difference between a covariance matrix created by an RBF kernel and a covariance matrix created by

I can't explain something simple to myself and it is probably a matter of vocabulary, I am not sure... If I create and random normal $Z \in \mathbb{R}^{3\times5}$, each row and column has a mean of 0. ...
2
votes
0answers
23 views

In the broadest sense, what is a "kernel"?

In MCMC sampling methods, a transition kernel, as found in Metropolis(/Hastings) algorithm, is the comparison of the likelihood of the current position and the likelihood of the proposed position. ...
0
votes
0answers
22 views

RBF networks and epsilon-NN

Will an RBF network with a Window kernel correspond to an epsilon-NN classifier? I know that kNN only considers the k nearest data points while RBF uses all the data. How does this relate to epsilon-...
2
votes
1answer
78 views

Prove that the following matrix is positive definite

We define $K_{\mathbf{a}, \mathbf{b}}$ as the $n \times m$ matrix whose $ij^{th}$ entry is $\kappa(a_{i}, b_{j})$ Where, $\kappa$ is a (positive definite) kernel function. Here, $\mathbf{a}_{i}, \...
0
votes
1answer
113 views

About Gaussian kernel for distances other than Euclidian

I have a question about Gaussian kernel. I read the following site. https://datascience.stackexchange.com/questions/17352/why-do-we-use-a-gaussian-kernel-as-a-similarity-metric My question is whether ...
1
vote
0answers
154 views

How to use sklearn's Gaussian Process Regression parameters?

I have been trying to play around with Gaussian process Regression. I have constructed a fake 1D data for this. I am using a Squared exponential kernel. I solved the regression problem using inbuilt ...
1
vote
0answers
33 views

How to extend kernel-based classifier to non-euclidean space like SO3

What is the proper way to extend kernel-based classifier to non-euclidean space like SO3? This kind of situation happens a lot in robotics, where the data points all live in a specific manifold. (Note:...
1
vote
0answers
22 views

Can someone improve my feeble understanding of feature maps and kernels?

I am taking the course and the extent in which we've discussed feature maps and kernels is as follows: Given Obviously we cannot use linear regression. Instead we map it to a space where it becomes ...
0
votes
1answer
86 views

Does radial basis function kernel has a coefficient?

I found there are two forms of RBF function. these is a coefficient before $\exp$ $$ k_{f}\left(x_{i}, x_{j}\right)=\sigma^{2} \exp \left(-\frac{1}{2 \ell^{2}} \...
0
votes
0answers
49 views

Kernel PCA stopping rule / criterion

I have been comparing dimensionality reduction methods, while performing KPCA, I didn't know what number of PCs I should retain, how many to take and decide : "there it is, this is the new ...
0
votes
0answers
21 views

Trying to use a custom kernel in SVR

I am very new to python and ML, I'm trying to customize a RBF kernel to consider Mahalanobis distance instead of Euclidean distance but I am encountering problems when I try to predict on new ...
1
vote
1answer
90 views

Calculation of Lipschiz constant for Square Exponential kernel

I am working with the Kernel function and want to calculate bounds using the concept of Lipschitz continuity. I do understand that the SE kernel is continuous, smooth, and differentiable. Is there any ...
2
votes
0answers
1k views

Why SVM with gamma='scale' for RBF kernel works so well?

The intuitive explanation for the gamma parameter of the RBF kernel in SVMs is the following: Intuitively, the gamma parameter ...
0
votes
0answers
20 views

Use cases exist for the Silverman kernel

I am well aware of the usage of the Gaussian, the boxcar function, the Epanechnikov function and others for use cases as kernel density estimation, Gaussian processes and others. But I have never seen ...
1
vote
1answer
437 views

Is it possible to find cluster centroids in kernel K means?

Suppose ${x_1, \ldots, x_N}$ are the data points and we have to find $K$ clusters using Kernel K Means. Let the kernel be $Ker$ (not to confuse with $K$ number of clusters) Let $\phi$ be the implicit ...
1
vote
0answers
62 views

Are radial basis kernels able to model interactions between predictors?

I have been doing research using Support Vector Regression for some time, especially using radial basis kernel, for predicting a response variable from a set of numeric predictors. As a consequence of ...
1
vote
1answer
90 views

SVR with combination of kernels

I am a beginner, and I am looking for some advice regarding the use of Support Vector Regression (SVR) to model (or fit if you prefer) a trend. Before you suggest other methods, for a number of ...
0
votes
2answers
52 views

Relationship between structural or statistical properties and hardness of classification

I am trying to understand the relationship between structural or statistical properties of training dataset and hardness of classification in the context of binary classification with SVM using RBF ...
1
vote
1answer
90 views

What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $k$ over any space $\mathcal{X}$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $k$. For example, Radial basis ...
1
vote
0answers
85 views

How is a polynomial kernel with infinite degree different from RBF Kernel?

I was reading about polynomial and RBF Kernels. According to my understanding: Polynomial kernels with degree >1 map the non-linear data into a higher dimensional feature space. Data that aren't ...
2
votes
1answer
965 views

Kernel approximation with Nystroem method and usage in scikit-learn

I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn. As far as I understood, the full Gram Matrix ...
0
votes
0answers
13 views

kernel and mean function for series of function

my series for functions are type. $$f(x) = a \sin(x-b) , a \sim \mathcal{N}(-1,2) , b \sim \mathcal{N}(-0.5,1)$$ Can someone get me started how to model these functions with GP. I am confused about ...
0
votes
1answer
76 views

Finding optimal kernel parameters

I want to perform multiple kernel learning on my dataset and apply each (rbf) kernel to a different subset of features to then combine them. I do not want to have the same kernel with a range of ...
0
votes
0answers
18 views

Applying different kernels to parts of a dataset and merging the output [duplicate]

I am trying to create a classifier using SVM on a dataset that is composed of 6 sets of data for each of my observations. When I train the SVM (rbf kernel), I get a better performance of the ...
0
votes
0answers
634 views

Calculate Gamma for RBF Kernel to get Gaussian Kernel

In order to measure the information density like proposed in section 3.2 of this paper I need a symmetric positive definite Kernel function. For this purpose I want to use the Gaussian Kernel like ...
1
vote
2answers
336 views

hyperparameters optimisation with linear kernel

I want to conduct an SVM model-regression (i.e., support vector regression), using a linear kernel function. Does it make sense to perform a cross-validation hyperparameter optimization when the ...
0
votes
1answer
227 views

Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
1
vote
0answers
138 views

What is the role of length scale bound in sklearn Radial Basis Function

The radial basis function provided by SkLearn (reference) has two parameters: length scale and length scale bounds. I understand that the length scale controls the importance of the coordinates of the ...
2
votes
1answer
107 views

What is the expanded representation, $\phi(X)$, required to obtain the RBF kernel?

For the two-dimensional case, where $\boldsymbol X=[x_1, x_2]$ and its corresponding expanded represetation $\boldsymbol\phi(X)= [1, \sqrt2 x_1, \sqrt2 x_2, x_1^2, x_2^2, \sqrt2x_1x_2]$, we can ...
0
votes
1answer
43 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
1
vote
0answers
20 views

Is it good idea to generate features from data points similarity comparison?

I know about polynomial features in machine learning, which can introduce nonlinearity to original dataset. I also heard about binning, which also allows us to create new features from existing ones. ...
13
votes
1answer
2k views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
1
vote
1answer
38 views

Is SVM RBF applied to both classes?

Lets say i have following 1D data (position on x), color is target class and I need a classifier which classifies green from red: I decided to use SVM. Data is clearly not linearly separable, so i ...
1
vote
0answers
432 views

Gaussian RBF vs KNN explanation

I was studying SVM ML alghorythm and I was wondering about solution for non-linear cases. As I understand it for know, SVM tries to find hyperplane or object in defined n-dimensional space, which ...
1
vote
0answers
48 views

Labeling KPCA in R

The function $\texttt{biplot}$ in R is very useful for creating visualizations when performing PCA. However, when performing kernel PCA in R, I cannot find a way to label the loadings on the graph. ...