Tagged Questions

Kernel refers to weighting functions used in non-parametric estimation techniques (such as kernel density estimation or kernel smoothing). DO NOT USE this tag for [kernel-trick] which is reserved for kernel methods in machine learning.

learn more… | top users | synonyms

0
votes
1answer
13 views

Has deconvolution been applied to nodes in a sensor network?

Convolution, and blind deconvolution, is generally applicable where there is some "truth" function describing a physical process, and then some kind of distortion. I have a bunch of sensors in a ...
0
votes
0answers
18 views

Selecting kernel or binary similarity measures

Currently, I am facing a choice of encoding some information either in a binary vector or a normalized (Gaussian) floating point vector of the same length. For instance it could be in the format [ 1, ...
0
votes
0answers
8 views

Kernel Methods for Binary Vectors

I am currently involved in a project which requires a minor point in choosing a proper similarity metric for a set of binary vectors, i.e. all components are either 1 or 0 . Currently, the go-to ...
-1
votes
0answers
8 views

kernel validation (is K=k1(x,t)-k2(x,t) a valid kernel? )

Suppose $k_1(x,t)=x't$ is a linear kernel and $k_2(k,t)=(x't)^2$ is a polynomial kernel. Is $K=k_1(x,t)-k_2(x,t)$ a valid kernel?
2
votes
0answers
30 views

“…if the data is linearly separable”

I keep hearing this phrase as a precursor to many algorithms, but I am not sure how exactly one goes about finding out if the data is indeed, linearly separable. Of course, if the data has ...
1
vote
2answers
49 views

Using a gaussian kernel in SVM. How exactly is this then written as a dot product?

I am attempting to use SVMs for my class project. For this project, I have selected the gaussian kernel as, well, the kernel. That is, $$ k(\mathbf{x}_1, \mathbf{x}_n) = e^{-\gamma ||\mathbf{x}_1 - ...
0
votes
0answers
16 views

Incorporating kernel into multiple regression

Let's say I have predictors $ \{x_1, x_2, ..., x_m, ... x_p\} $. I want to fit a multiple regression using $\{x_1,...,x_m\}$, but give more weight to points that are close to a particular $\vec{x}^*$ ...
0
votes
0answers
27 views

My support vectors don't look correct

I am trying to classify a toy dataset using SVM. I only have two features and 20 instances. The decision boundary seems correct, however, the support vectors dont look correct. This is the relevant ...
0
votes
0answers
11 views

r-pnn, normalization and different distance measures for each variable

Since pnn is a NN that uses a Radial kernel to classify data, I think the distance measure is key and, in consequence, the normalization of the data. Am I right? How does pnn package calculate the ...
0
votes
0answers
26 views

Kernel density estimation on bounded support

I was looking for some way to deal with boundary bias of kde in case of unit interval. One example is an usage of Chen estimators (or Beta estimators; an example might be seen here: ...
0
votes
0answers
25 views

Merging two disconnected graphs

Firstly, I'd like to apologize for any misused terms or ways I could have made the description much more succinct. It's been a while since I took machine learning during my bachelor's. I have two ...
0
votes
0answers
18 views

SVM with pre-computed kernel and zero bias

I have an optimization function, where I need to give my own kernel matrix and bias value is zero. The kernel matrix is calculated using the data but there is no specific formula for it. If I have a ...
0
votes
0answers
18 views

LIBSVM for pre-computed kernel and zero bias (b values is zero)

I want to do binary classification and I'm using LIBSVM library for that. I have a precomputed Kernel and my bias value (b) is zero. Can I do this in LIBSVM or do I have to use some other library? ...
0
votes
0answers
18 views

Usage of libsvm with RBF kernel and no Offset

I'm using libsvm for the binary classification and using a precomputed Kernel. In my particular problem there is no bias term (it's zero). Is there anyway to adjust the bias term in libsvm (and not ...
2
votes
1answer
20 views

How to choose a kernel for KDE

There are a lot of kernels available for a univariate KDE. R uses normal by default, but the efficacy discussion seems to support the use of Epanechnikov. What should influence kernel choice for ...
0
votes
1answer
22 views

What is the toolkit that implements Cost sensitive Support Vector Machine?

I need implementation of cost sensitive support vector machine. The cost is different for each training example (unlike each class). So problem is to solve $max_\alpha$ $-1/2 \sum_{i,j} ...
0
votes
0answers
25 views

Use cases for P-Kernel for SVMs

I've been reading the book by Cristianini on Kernels (2004) where generative kernels (like p-kernel and fisher-kernel, not to be confused with polynomial kernel!) are described. I am interested in ...
0
votes
0answers
6 views

Can a kernel function for GP-regression use measurement information?

when building a kernel function for a Gaussian-Process-Regression I am asking myself whether the kernel function is allowed to contain information from the measurements. To ask a little more general, ...
1
vote
1answer
26 views

References to papers/books that uses a kernel to smooth a discrete distribution

Since a kernel, such as Gaussian, is often used to smooth out the distribution of discrete points in 1D, 2D or 3D, I believe there must be some study materials or research work that have used this, ...
1
vote
1answer
79 views

why simulated gamma distributed data have negative kernel values?

I know that Gamma distribution does not allow 0 or negative values. I was doing some simulation and when I write this code in R ...
0
votes
0answers
24 views

How are local and global plug-in bandwidths different in kernel smoothing regression?

I'm looking into an R package 'lokern.' It provides two bandwidths selectors, global and local plug-in bandwidth. I would like to understand the difference between two methods. My understanding is ...
0
votes
0answers
22 views

Covariance vs Bandwidth of Kernel Density Estimate

I've been working with the scipy gaussian kde implementation (here), but I don't quite understand the difference between the bandwidth factor and the covariance matrix. I'm using it for a single ...
1
vote
1answer
147 views

What are basic differences between Kernel Approaches to Unsupervised and Supervised Machine Learning

I got nice graphical representation of Machine learning for clustering / classification. Source: Kernel Approaches to Unsupervised and Supervised Machine Learning by Sun-Yuan Kung Here are my ...
4
votes
2answers
82 views

When do kernel based method perform better than the regular

I am used with linear models. I can see rising use of kernel based method particularly in machine learning. The following is an example Gaussian kernel using ...
0
votes
1answer
83 views

kernels and similarity (in R)

I am trying fit different kernels to calculate similarity matrix in R. Here is example data - X matrix : ...
0
votes
0answers
14 views

Parameter estimation of gaussian function kernel using cross-validation

I need to estimate (using cross-validation), the parameters $\sigma$ and $\lambda$ of the Gaussian kernel: $K_G(x,y) = \sigma^2 \exp{(-\frac{1}{2\lambda^2}\sum_{i,j}(x_{ij}-y_{ij})^2})$ where $x$ ...
1
vote
1answer
37 views

Applying an RBF kernel first and then train using a Linear Classifier

I will start off by saying that I don't have a concrete understanding of whats under the hood of a SVM classifier. I am interested in using an SVM with the RBF kernel to train a two class ...
2
votes
1answer
89 views

How to get percentiles from empirical density in R?

The density() function in R allows me to enter observations and get an empirical density that I can plot x and y values. I like ...
1
vote
1answer
68 views

Where can I use kernels other than Gaussian (like Cauchy, laplacian) in kernel methods in machine learning? Or maybe in kernel density estimation?

In few papers I read that - kernel used doesn't really matter for kernel density estimation but bandwidth of the kernel is the most important factor. But I did not see any mathematical explanation to ...
0
votes
0answers
5 views

How accurate sum of kernel function needs to be, so that we can use it in Mean shift algorithm (may be for image segmentation)?

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. It is useful for detecting the modes of this density. This is an iterative ...
0
votes
0answers
70 views

Spectral clustering using RBF Kernel function in R

I have extracted user-features and item features in my recommender system using a modified SVD approach built on ALSE (loosely based on Yehuda Koren's paper). I now want to cluster items not directly ...
1
vote
1answer
99 views

Extracting decision function variable from libsvm

I'm trying to use LIBSVM's single class SVMs for some classification and need to extract the following sum post classification (i.e. the variable that the decision function takes in) $$ ...
1
vote
0answers
53 views

STRING KERNELS FOR LIBSVM

I'm working on a protein classification problem and i'm using edit distance kernel defined in libSVM. Now, for instance, the implementation of spectrum kernel is very difficult, but i want try to test ...
0
votes
2answers
43 views

Evolution strategies in libsvm

I'm working on protein multi-classification problem. I'm using libsvm and the edit distance kernel. This kernel depends from a parameter (gamma). I'm able to get the best parameters (gamma and C) ...
4
votes
3answers
111 views

Do kernel methods “scale” with the amount of data?

I've been reading about kernel methods, where you map original $N$ data points to a feature spaces, compute the kernel or gram matrix and plug that matrix into a standard, linear algorithm. This all ...
0
votes
1answer
69 views

Improving SVM classification

I have a classification problem (bioinformatics domain) where I have around 333 features. Currently, I am first selecting features (using importance feature of random forest) and then pushing the same ...
0
votes
0answers
10 views

In RVM, is the kernel allowed to depend on the full dataset?

I want to use Relevance Vector Machines and need to define my custom made kernel. I was wondering if it is allowed for the kernel to depend on the full dataset. For exampe, I can calculate a certain ...
1
vote
0answers
16 views

construct/load dataset that performs better with diffusion kernel than other kernel

I'm looking for a dataset on which a diffusion kernel (also called heat kernel), used via SVM, would get better accuracy than other kernels for the classification task. I want to use such a dataset to ...
0
votes
0answers
54 views

Linear Kernel taking more time to train than RBF Kernel (SVR)

I'm doing a Support Vector Regression with about 70k samples with 500 features each. I'm using sklearn implementation of SVR and my input for the train set is a sparse matrix. But, for my surprise, ...
3
votes
1answer
105 views

Understand the reasons of using Kernel method in SVM

I understand that one can use kernel functions (i.e. radial kernel) to create non-linear decision boundary. However, there is something with my logic and I am sure there is something that I clearly ...
1
vote
1answer
37 views

Local log-likelihood for multiclass linear regression model

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
1
vote
1answer
99 views

Kernel density estimator function explanation needed

I'm studying about kernel density estimation and from wikipedia I get this formula: $$\hat{f}_h(x, h) = \frac{1}{n}\sum^{n}_{i=1}K_h(x-x_i) = \frac{1}{nh}\sum_{i=1}^nK(\frac{x-x_i}{h}).$$ I think ...
0
votes
1answer
74 views

Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
0
votes
1answer
121 views

Accuracy changes with permutation of input data in Libsvm with precomputed kernel?

I'm doing quite simple SVM classification at the moment. I use a precomputed kernel in LibSVM with RBF and DTW. When I compute the similarity (kernel-) matrix, everything seems to work very fine ... ...
3
votes
1answer
78 views

Efficient evaluation of multidimensional kernel density estimate

I've seen a reasonable amount of literature about how to choose kernels and bandwidths when computing a kernel density estimate, but I am currently interested in how to improve the time it takes to ...
0
votes
0answers
120 views

What's wrong with my Kernel algorithm (Kernel SVD)?

I have a user-item matrix $A$ as data input, which is a sparse matrix containing a large number of missing values (as zeros). Each row is a user, and each column is an item. Generally, I am conducting ...
1
vote
0answers
28 views

Regression of family of marginal density functions

From a large sample of triples $(X, Y, U)$ I need to estimate a function $(x, y, u) \mapsto f(x, y, u)$, such that for each fixed $x, y$, the function $u \mapsto f(x, y, u)$ is a density function; ...
2
votes
0answers
110 views

Prove that this exponential kernel is positive definite

Let $x,y\in R^d$ and $d:R^d\times R^d \rightarrow R$ a metric on $R^d$ be given. The exponential kernel is defined by: $k(x,x')=e^{−αd(x,x')}$ where $α>0$. The kernel matrix is defined as the ...
1
vote
1answer
65 views

Cascade Combination of Kernel Functions

I have a question regarding machine learning and specifically kernel functions. Suppose we have a Kernel function, say $K(x)$, and also another distinct one, say $K'(x)$. I want to know is $K(K'(x))$ ...
1
vote
0answers
41 views

Difference between Primal, Dual and Kernel Ridge Regression

I would like to basically ask what the title says. What is the difference between Primal, Dual and Kernel Ridge Regression? People are using all three, and because of the different notation that ...