Questions tagged [rbf-kernel]

The RBF kernel, i.e., radial-basis-function kernel, occurs in the context of kernel methods in machine learning.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Relationship between structural or statistical properties and hardness of classification

I am trying to understand the relationship between structural or statistical properties of training dataset and hardness of classification in the context of binary classification with SVM using RBF ...
1
vote
1answer
24 views

What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $k$ over any space $\mathcal{X}$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $k$. For example, Radial basis ...
1
vote
0answers
20 views

How is a polynomial kernel with infinite degree different from RBF Kernel?

I was reading about polynomial and RBF Kernels. According to my understanding: Polynomial kernels with degree >1 map the non-linear data into a higher dimensional feature space. Data that aren't ...
0
votes
0answers
5 views

heuristics for gamma in rbf kernel

My question is a follow-up to this question: SVM rbf kernel - heuristic method for estimating gamma. Basically, I want to find interesting values for gamma by first calculating the pairwise distance ...
0
votes
0answers
17 views

Can I customize the kernel function?

I want to know whether I can customize the kernel function? For example, the polynomial kernel is defined as: $$ K(x,y) = (x^Ty+c)^d $$ Could I modify it to the following: $$ K(x,y) = (||x-y||_2)^d $$ ...
0
votes
1answer
48 views

Kernel approximation with Nystroem method and usage in scikit-learn

I am planning to use the Nystroem method to approximate a Gram matrix induced by any kernel function. I found the Nystroem implementation in scikit-learn. As far as I understood, the full Gram Matrix ...
0
votes
0answers
12 views

kernel and mean function for series of function

my series for functions are type. $$f(x) = a \sin(x-b) , a \sim \mathcal{N}(-1,2) , b \sim \mathcal{N}(-0.5,1)$$ Can someone get me started how to model these functions with GP. I am confused about ...
0
votes
1answer
30 views

Finding optimal kernel parameters

I want to perform multiple kernel learning on my dataset and apply each (rbf) kernel to a different subset of features to then combine them. I do not want to have the same kernel with a range of ...
0
votes
0answers
16 views

Applying different kernels to parts of a dataset and merging the output [duplicate]

I am trying to create a classifier using SVM on a dataset that is composed of 6 sets of data for each of my observations. When I train the SVM (rbf kernel), I get a better performance of the ...
0
votes
0answers
27 views

Using Gaussian Process Regression in scikit-learn

I have a simple dataset with multiple trials of position over time, and I'm trying to fit a Gaussian Process over it. Here's a plot of all the raw data (6180 data points): My goal is to fit a ...
0
votes
0answers
20 views

Using RBF k-NN graph in spectral clustering

In the article Spectral Clustering with Imbalanced Data there is mentioned usage of a "RBF k-NN" graph. I haven't encountered this kind of graphs before and couldn't google anything related to it. ...
0
votes
0answers
33 views

Gaussian Process Regression for High Dimensional Data: Vanishing predicted y values

I am working on a dataset with roughly 196 dimensions. I have been trying to fit Gaussian Process Regression into this dataset but it does not perform well. Mathematically speaking, I found out that ...
0
votes
0answers
129 views

Calculate Gamma for RBF Kernel to get Gaussian Kernel

In order to measure the information density like proposed in section 3.2 of this paper I need a symmetric positive definite Kernel function. For this purpose I want to use the Gaussian Kernel like ...
0
votes
0answers
51 views

Implementation of a Gauss Kernel in Python possibly using RBF Client

I want to implement the following Gauss kernel in Python: I could implement the structure in Python up to this point. However, the last piece missing is the calculation of the parameter tau squared. ...
0
votes
0answers
20 views

Resource for understanding kernel trick, kernel method, kernel functions?

So far my understanding about kernel methods is that they are ways to map our features to a higher dimension space - allowing us to fit non-linear data using linear models. I don't understand much ...
1
vote
2answers
39 views

hyperparameters optimisation with linear kernel

I want to conduct an SVM model-regression (i.e., support vector regression), using a linear kernel function. Does it make sense to perform a cross-validation hyperparameter optimization when the ...
0
votes
1answer
37 views

Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
1
vote
0answers
51 views

What is the role of length scale bound in sklearn Radial Basis Function

The radial basis function provided by SkLearn (reference) has two parameters: length scale and length scale bounds. I understand that the length scale controls the importance of the coordinates of the ...
0
votes
0answers
12 views

Why do Support Vector Data Description and One Class Support Vector Machine produce the same results?

Quoating from Chapter 5 of Kernel Methods in Computer Vision by Christoph H. Lampert 'A quick geometric check shows that if all data points have the same feature space norm and can be separated ...
1
vote
1answer
39 views

What is the expanded representation, $\phi(X)$, required to obtain the RBF kernel?

For the two-dimensional case, where $\boldsymbol X=[x_1, x_2]$ and its corresponding expanded represetation $\boldsymbol\phi(X)= [1, \sqrt2 x_1, \sqrt2 x_2, x_1^2, x_2^2, \sqrt2x_1x_2]$, we can ...
0
votes
0answers
62 views

Relation between choice of kernel for the affinity matrix in spectral clustering and embedding into a higher dimensional space, using feature maps

So I've been studying spectral clustering where they use some affinity function related to a pre-constructed graph of the sample points or data $\{x_1,...x_n\}$. If we call the affinity function $W$, ...
0
votes
1answer
31 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
0
votes
0answers
53 views

Binary classification Task : Least Squares kernel regression(squared loss) Vs SVM (hinge loss)

In binary classification, the solution function, in order to fit the training data, it just needs to acquire values that have the same polarity as the desired values, rather than accurately acquiring ...
1
vote
0answers
20 views

Is it good idea to generate features from data points similarity comparison?

I know about polynomial features in machine learning, which can introduce nonlinearity to original dataset. I also heard about binning, which also allows us to create new features from existing ones. ...
10
votes
1answer
599 views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
1
vote
1answer
33 views

Is SVM RBF applied to both classes?

Lets say i have following 1D data (position on x), color is target class and I need a classifier which classifies green from red: I decided to use SVM. Data is clearly not linearly separable, so i ...
1
vote
0answers
117 views

Gaussian RBF vs KNN explanation

I was studying SVM ML alghorythm and I was wondering about solution for non-linear cases. As I understand it for know, SVM tries to find hyperplane or object in defined n-dimensional space, which ...
0
votes
0answers
5 views

RBF-Kernel: Handling missing values

I want to compute the RBF-Kernel for a dataset which contains missing values: ...
1
vote
0answers
14 views

Labeling KPCA in R

The function $\texttt{biplot}$ in R is very useful for creating visualizations when performing PCA. However, when performing kernel PCA in R, I cannot find a way to label the loadings on the graph. ...
1
vote
1answer
73 views

Gaussian processes Sum of RBF kernels vs single anisotropic RBF kernel [closed]

Say I have some two dimensional data, for which I am trying to fit a Gaussian process. In scikit-learn, I can build an RBF kernel as follows K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + ...
3
votes
1answer
55 views

The inner product properties seem to clash with the RKHS property for RBF kernels. What is off?

By the reproducing kernel Hilbert space (RKHS) property, given a P.S.D. kernel function $\kappa:X\times X \rightarrow \mathbb R$, there exists a Hilbert space $H$ and a map $\phi:X\rightarrow H$ such ...
0
votes
0answers
19 views

SVM and RBF Kernel

I have read that high gamma value in SVM(rbf kernel) can lead to high bias. But I have seen high gamma overfits the decision boundary. Why is it not called high variance?
0
votes
0answers
35 views

Significance of initialisation of Kernel in sklearn.gaussian_process.kernels

I have been going through Gaussian Processes. In one of the code I stumbled upon there is this statement, I am not quite sure of the parameters that are passed to initialise it. Please help me. ...
1
vote
1answer
57 views

Fundamental understanding of Gaussian Process and their terminology [closed]

I am new to this site as well as Machine learning, so kindly bear with me. I have been trying to understand Gaussian process and their implementation. Notation: 1) Let's say that the $\vec{x}$ $\in ...
0
votes
2answers
226 views

RBF kernel mapping

I was reading that the Gaussian/RBF kernel maps its input onto the surface of normalized hypersphere. Our RBF kernel given by: $k(x,z) = exp(\frac{- ||x-z||^2}{2\sigma^2})$ Can anyone explain why ...
2
votes
0answers
194 views

Kernel function with a feature space equipped with an inner product that is not the dot product

Premise: A function $K: \mathbb R^d \times \mathbb R^d \to \mathbb R$ is called a kernel function on $\mathbb{R}^d$ if there exists a Hilbert space $\mathcal{H}$ and a map $\phi: \mathbb R^d \...
1
vote
0answers
39 views

Possible error in evaluating kernel gradient in scikit-learn's GPR

Perhaps I am missing something very obvious, but in the standard kernels associated with scikit-learn's Gaussian process regression framework, the radial basis function (RBF), $$f = e^{-x^2/2l^2},$$ ...
1
vote
1answer
96 views

Is this a valid Gaussian Process kernel?

$\mathcal{K}\Big( \; (x,y), (x',y') \; \Big) = \sigma_f^2 \exp{ \frac{(x-x')^2}{2l^2 \cdot (y+y')^2} } $, where $l > 0$ The variance associated with each training point (given by a vector) is a ...
2
votes
0answers
223 views

What is the most intuitive proof that Gaussian kernel is positive definite? [duplicate]

I have general form of Gaussian kernel $K(x,x')=\exp(-\|x-x'\|^{2})$ (just not considering $\sigma$). I tried to prove its positive definiteness via Gram matrix properties, but couldn't. Is there any ...
8
votes
1answer
245 views

Convergence of the Matérn covariance function to the squared exponential

The Matérn covariance function converges to the squared exponential covariance function. Many sources, amongst them the GPML book and Wikipedia, state this result. None of them provide details. I ...
1
vote
1answer
241 views

Probabilistic Interpretation of Radial Basis Function

I was wondering if someone could flesh out the probabilistic interpretation of using the Radial Basis Function to compute the probability between an observation and some reference value. My question ...
0
votes
1answer
247 views

SVM and correlation

Can anyone guide me about the feature selection based on correlation using SVM? RBF kernel check the correlation too or not? I am using weka and matlab. Any help would be appreciated.
1
vote
1answer
716 views

Is this a valid kernel function?

I devised a distance function similar to this form. K(x,y)=(-||x-y||+xy+1)/2 And now I want to prove it is a kernel function.I have read about Mercer's condition and positive semi definiteness, but I ...
1
vote
0answers
79 views

Use the features selected with RFE SVM linear for prediction of SVM rbf

I was wondering if the features selected with RFE with SVM linear kernel are still "good" features when we use a non linear model, like SVM rbf kernel. This comes in practice when you want to use SVM ...
1
vote
1answer
467 views

How to perform kFold cross validation in Libsvm's precomputed kernel in MATLAB?

I understand that Libsvm provides 'v 10' option for 10-fold cross-validation in...
1
vote
1answer
70 views

Kernel functions with vector output

Kernel functions are used commonly with SVMs to make classification of non linearly separable data possible - i.e. the Kernel function provides the linear separability. But from looking at Kernel ...
1
vote
0answers
43 views

Adding new center in an RBF network without memorizing previous training examples

Suppose we train an RBF by minimizing the LSE on a couple of training points and we are doing it incrementally in an online fashion. So basically we update the QR factorization using e.g. Givens ...
2
votes
1answer
806 views

linear vs non-linear kernel SVM

The dataSet contains 213 examples of 7 classes . Each example are 25000 features. I want to learn model with SVM (test scenario used are 10-fold cross validation). I am a beginner in machine learning, ...
2
votes
1answer
290 views

What is the motivation or objective for adopting Kernel methods? Is kernel trick a feature engineering method?

I come to know that kernel methods can be used in not only SVM but also many machine learning algorithms. I understanding that in SVM, the reason for using kernel trick is that some data are linearly ...
1
vote
0answers
99 views

Kernels property: integral of kernel product $\propto k(x,y)$

Let $k$ be a kernel function (symmetric and semi-positive definite function). Does the following relationship hold: $\int_{-\infty}^{+\infty}k(x,u)k(y,u) du \propto k(x,y)$ ? Or for what type of ...