Questions tagged [rbf-kernel]

The RBF kernel, i.e., radial-basis-function kernel, occurs in the context of kernel methods in machine learning.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Gaussian Process Regression for High Dimensional Data: Vanishing predicted y values

I am working on a dataset with roughly 196 dimensions. I have been trying to fit Gaussian Process Regression into this dataset but it does not perform well. Mathematically speaking, I found out that ...
0
votes
0answers
17 views

Calculate Gamma for RBF Kernel to get Gaussian Kernel

In order to measure the information density like proposed in section 3.2 of this paper I need a symmetric positive definite Kernel function. For this purpose I want to use the Gaussian Kernel like ...
0
votes
0answers
10 views

Implementation of a Gauss Kernel in Python possibly using RBF Client

I want to implement the following Gauss kernel in Python: I could implement the structure in Python up to this point. However, the last piece missing is the calculation of the parameter tau squared. ...
0
votes
0answers
10 views

Resource for understanding kernel trick, kernel method, kernel functions?

So far my understanding about kernel methods is that they are ways to map our features to a higher dimension space - allowing us to fit non-linear data using linear models. I don't understand much ...
0
votes
1answer
172 views

SVM and correlation

Can anyone guide me about the feature selection based on correlation using SVM? RBF kernel check the correlation too or not? I am using weka and matlab. Any help would be appreciated.
1
vote
2answers
22 views

hyperparameters optimisation with linear kernel

I want to conduct an SVM model-regression (i.e., support vector regression), using a linear kernel function. Does it make sense to perform a cross-validation hyperparameter optimization when the ...
0
votes
0answers
17 views

Relation between Bayesian Linear Regression (Fixed base: Gaussian RBF) and SVM RBF?

I am trying to get my head around Bayesian Linear Regression. I am looking at a Gaussian radial basis function, which I assume acts as our prior. I have the following diagram: My current ...
0
votes
1answer
23 views

Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
1
vote
0answers
19 views

What is the role of length scale bound in sklearn Radial Basis Function

The radial basis function provided by SkLearn (reference) has two parameters: length scale and length scale bounds. I understand that the length scale controls the importance of the coordinates of the ...
0
votes
0answers
8 views

Why do Support Vector Data Description and One Class Support Vector Machine produce the same results?

Quoating from Chapter 5 of Kernel Methods in Computer Vision by Christoph H. Lampert 'A quick geometric check shows that if all data points have the same feature space norm and can be separated ...
0
votes
0answers
30 views

Relation between choice of kernel for the affinity matrix in spectral clustering and embedding into a higher dimensional space, using feature maps

So I've been studying spectral clustering where they use some affinity function related to a pre-constructed graph of the sample points or data $\{x_1,...x_n\}$. If we call the affinity function $W$, ...
0
votes
1answer
33 views

What is the expanded representation, $\phi(X)$, required to obtain the RBF kernel?

For the two-dimensional case, where $\boldsymbol X=[x_1, x_2]$ and its corresponding expanded represetation $\boldsymbol\phi(X)= [1, \sqrt2 x_1, \sqrt2 x_2, x_1^2, x_2^2, \sqrt2x_1x_2]$, we can ...
0
votes
1answer
26 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
0
votes
0answers
29 views

Binary classification Task : Least Squares kernel regression(squared loss) Vs SVM (hinge loss)

In binary classification, the solution function, in order to fit the training data, it just needs to acquire values that have the same polarity as the desired values, rather than accurately acquiring ...
8
votes
1answer
250 views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
1
vote
0answers
18 views

Is it good idea to generate features from data points similarity comparison?

I know about polynomial features in machine learning, which can introduce nonlinearity to original dataset. I also heard about binning, which also allows us to create new features from existing ones. ...
1
vote
1answer
26 views

Is SVM RBF applied to both classes?

Lets say i have following 1D data (position on x), color is target class and I need a classifier which classifies green from red: I decided to use SVM. Data is clearly not linearly separable, so i ...
1
vote
0answers
69 views

Gaussian RBF vs KNN explanation

I was studying SVM ML alghorythm and I was wondering about solution for non-linear cases. As I understand it for know, SVM tries to find hyperplane or object in defined n-dimensional space, which ...
0
votes
0answers
4 views

RBF-Kernel: Handling missing values

I want to compute the RBF-Kernel for a dataset which contains missing values: ...
1
vote
0answers
11 views

Labeling KPCA in R

The function $\texttt{biplot}$ in R is very useful for creating visualizations when performing PCA. However, when performing kernel PCA in R, I cannot find a way to label the loadings on the graph. ...
1
vote
1answer
42 views

Gaussian processes Sum of RBF kernels vs single anisotropic RBF kernel [closed]

Say I have some two dimensional data, for which I am trying to fit a Gaussian process. In scikit-learn, I can build an RBF kernel as follows K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + ...
2
votes
1answer
48 views

The inner product properties seem to clash with the RKHS property for RBF kernels. What is off?

By the reproducing kernel Hilbert space (RKHS) property, given a P.S.D. kernel function $\kappa:X\times X \rightarrow \mathbb R$, there exists a Hilbert space $H$ and a map $\phi:X\rightarrow H$ such ...
0
votes
0answers
18 views

SVM and RBF Kernel

I have read that high gamma value in SVM(rbf kernel) can lead to high bias. But I have seen high gamma overfits the decision boundary. Why is it not called high variance?
0
votes
2answers
149 views

RBF kernel mapping

I was reading that the Gaussian/RBF kernel maps its input onto the surface of normalized hypersphere. Our RBF kernel given by: $k(x,z) = exp(\frac{- ||x-z||^2}{2\sigma^2})$ Can anyone explain why ...
0
votes
0answers
24 views

Significance of initialisation of Kernel in sklearn.gaussian_process.kernels

I have been going through Gaussian Processes. In one of the code I stumbled upon there is this statement, I am not quite sure of the parameters that are passed to initialise it. Please help me. ...
2
votes
0answers
110 views

Kernel function with a feature space equipped with an inner product that is not the dot product

Premise: A function $K: \mathbb R^d \times \mathbb R^d \to \mathbb R$ is called a kernel function on $\mathbb{R}^d$ if there exists a Hilbert space $\mathcal{H}$ and a map $\phi: \mathbb R^d \...
1
vote
1answer
40 views

Fundamental understanding of Gaussian Process and their terminology [closed]

I am new to this site as well as Machine learning, so kindly bear with me. I have been trying to understand Gaussian process and their implementation. Notation: 1) Let's say that the $\vec{x}$ $\in ...
2
votes
0answers
211 views

What is the most intuitive proof that Gaussian kernel is positive definite? [duplicate]

I have general form of Gaussian kernel $K(x,x')=\exp(-\|x-x'\|^{2})$ (just not considering $\sigma$). I tried to prove its positive definiteness via Gram matrix properties, but couldn't. Is there any ...
1
vote
1answer
438 views

Is this a valid kernel function?

I devised a distance function similar to this form. K(x,y)=(-||x-y||+xy+1)/2 And now I want to prove it is a kernel function.I have read about Mercer's condition and positive semi definiteness, but I ...
0
votes
0answers
31 views

How would phi of the gaussian rbf kernel map a 100-by-3 dimensional feature matrix?

Would a 100-by-3 dimensional feature matrix be mapped into a 100 dimensional or into a infinite dimensional feature space, if the mapping would not be bypassed by the Gaussian RBF Kernel? Following ...
0
votes
1answer
2k views

Gamma as inverse of the variance of RBF kernel

I would like to fix the parameter gamma by using the following heuristic and then select C using GridSearch: taking the inverse of variance of RBF. ...
0
votes
0answers
28 views

Reproducing kernels: how do I numerically compute the decomposition?

Suppose I'm given a kernel, $$ K(x,y) : \mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R} $$ In order to describe/understand the (unique) associated RKHS, I seek its eigenfunctions, as per Mercer'...
0
votes
0answers
87 views

R - Gamma estimates in Kernel Ridge Regression

I am running a Kernel Ridge Regression in R. Mathematically, the minimization problem to be solved is the following: $$ \min_{\boldsymbol{\beta} \in \mathbb{R}^{d}} \ \sum_{i = 1}^{n} (y_{i} - \left \...
1
vote
0answers
34 views

Possible error in evaluating kernel gradient in scikit-learn's GPR

Perhaps I am missing something very obvious, but in the standard kernels associated with scikit-learn's Gaussian process regression framework, the radial basis function (RBF), $$f = e^{-x^2/2l^2},$$ ...
1
vote
1answer
84 views

Is this a valid Gaussian Process kernel?

$\mathcal{K}\Big( \; (x,y), (x',y') \; \Big) = \sigma_f^2 \exp{ \frac{(x-x')^2}{2l^2 \cdot (y+y')^2} } $, where $l > 0$ The variance associated with each training point (given by a vector) is a ...
1
vote
1answer
437 views

Is Bias term required for RBF with Gaussian kernel?

For standard logistic regression, we add a bias term (1) in the features. Is it required when RBF is used with Gaussian Kernel?
7
votes
1answer
208 views

Convergence of the Matérn covariance function to the squared exponential

The Matérn covariance function converges to the squared exponential covariance function. Many sources, amongst them the GPML book and Wikipedia, state this result. None of them provide details. I ...
1
vote
1answer
185 views

Probabilistic Interpretation of Radial Basis Function

I was wondering if someone could flesh out the probabilistic interpretation of using the Radial Basis Function to compute the probability between an observation and some reference value. My question ...
1
vote
0answers
63 views

Use the features selected with RFE SVM linear for prediction of SVM rbf

I was wondering if the features selected with RFE with SVM linear kernel are still "good" features when we use a non linear model, like SVM rbf kernel. This comes in practice when you want to use SVM ...
1
vote
1answer
387 views

How to perform kFold cross validation in Libsvm's precomputed kernel in MATLAB?

I understand that Libsvm provides 'v 10' option for 10-fold cross-validation in...
1
vote
0answers
43 views

Adding new center in an RBF network without memorizing previous training examples

Suppose we train an RBF by minimizing the LSE on a couple of training points and we are doing it incrementally in an online fashion. So basically we update the QR factorization using e.g. Givens ...
1
vote
1answer
51 views

Kernel functions with vector output

Kernel functions are used commonly with SVMs to make classification of non linearly separable data possible - i.e. the Kernel function provides the linear separability. But from looking at Kernel ...
2
votes
1answer
726 views

linear vs non-linear kernel SVM

The dataSet contains 213 examples of 7 classes . Each example are 25000 features. I want to learn model with SVM (test scenario used are 10-fold cross validation). I am a beginner in machine learning, ...
2
votes
1answer
223 views

What is the motivation or objective for adopting Kernel methods? Is kernel trick a feature engineering method?

I come to know that kernel methods can be used in not only SVM but also many machine learning algorithms. I understanding that in SVM, the reason for using kernel trick is that some data are linearly ...
1
vote
0answers
87 views

Kernels property: integral of kernel product $\propto k(x,y)$

Let $k$ be a kernel function (symmetric and semi-positive definite function). Does the following relationship hold: $\int_{-\infty}^{+\infty}k(x,u)k(y,u) du \propto k(x,y)$ ? Or for what type of ...
3
votes
0answers
53 views

Kernel and regularization parameter of James–Stein estimator

Consider a FIR model of the form $y= Ug_0+e$ with $e$ white noise with variance $\sigma^2$. We assume that we have collected N input-output measurements $y$ and $U$. The James–Stein estimator is ...
11
votes
2answers
296 views

Does Mercer's theorem work in reverse?

A colleague has a function $s$ and for our purposes it is a black-box. The function measures the similarity $s(a,b)$ of two objects. We know for sure that $s$ has these properties: The similarity ...
0
votes
1answer
321 views

How to tune bandwidth in machine learning kernel model?

Gaussian kernel $k(x,y) = \exp(-\lVert x-y \rVert^2/\sigma^2)$ has a hyperparameter $\sigma$. I know grid search cross validation, but this would require a lot of computation since computational ...
1
vote
1answer
451 views

How to understand effect of RBF kernel for kernel PCA

I understand the math in kernel PCA and with RBF kernel, and I also understand that the RBF kernel map the data into a infinite dimensional space. I know that for SVM, mapping the data into a higher ...
1
vote
0answers
173 views

The gamma parameter (or kernel width) for RBF Gaussian kernel in kernel PCA

Is there any general way or rule of thumb of how to determine the kernel width for KPCA?