Kernel refers to weighting functions used in non-parametric estimation techniques.

learn more… | top users | synonyms

2
votes
1answer
39 views

How to get percentiles from empirical density in R?

The density() function in R allows me to enter observations and get an empirical density that I can plot x and y values. I like ...
1
vote
1answer
38 views

Where can I use kernels other than Gaussian (like Cauchy, laplacian) in kernel methods in machine learning? Or maybe in kernel density estimation?

In few papers I read that - kernel used doesn't really matter for kernel density estimation but bandwidth of the kernel is the most important factor. But I did not see any mathematical explanation to ...
0
votes
0answers
5 views

How accurate sum of kernel function needs to be, so that we can use it in Mean shift algorithm (may be for image segmentation)?

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. It is useful for detecting the modes of this density. This is an iterative ...
0
votes
0answers
22 views

Spectral clustering using RBF Kernel function in R

I have extracted user-features and item features in my recommender system using a modified SVD approach built on ALSE (loosely based on Yehuda Koren's paper). I now want to cluster items not directly ...
1
vote
1answer
47 views

Extracting decision function variable from libsvm

I'm trying to use LIBSVM's single class SVMs for some classification and need to extract the following sum post classification (i.e. the variable that the decision function takes in) $$ ...
1
vote
0answers
22 views

STRING KERNELS FOR LIBSVM

I'm working on a protein classification problem and i'm using edit distance kernel defined in libSVM. Now, for instance, the implementation of spectrum kernel is very difficult, but i want try to test ...
0
votes
2answers
30 views

Evolution strategies in libsvm

I'm working on protein multi-classification problem. I'm using libsvm and the edit distance kernel. This kernel depends from a parameter (gamma). I'm able to get the best parameters (gamma and C) ...
4
votes
3answers
91 views

Do kernel methods “scale” with the amount of data?

I've been reading about kernel methods, where you map original $N$ data points to a feature spaces, compute the kernel or gram matrix and plug that matrix into a standard, linear algorithm. This all ...
0
votes
1answer
63 views

Improving SVM classification

I have a classification problem (bioinformatics domain) where I have around 333 features. Currently, I am first selecting features (using importance feature of random forest) and then pushing the same ...
0
votes
0answers
10 views

In RVM, is the kernel allowed to depend on the full dataset?

I want to use Relevance Vector Machines and need to define my custom made kernel. I was wondering if it is allowed for the kernel to depend on the full dataset. For exampe, I can calculate a certain ...
1
vote
0answers
15 views

construct/load dataset that performs better with diffusion kernel than other kernel

I'm looking for a dataset on which a diffusion kernel (also called heat kernel), used via SVM, would get better accuracy than other kernels for the classification task. I want to use such a dataset to ...
0
votes
0answers
17 views

Linear Kernel taking more time to train than RBF Kernel (SVR)

I'm doing a Support Vector Regression with about 70k samples with 500 features each. I'm using sklearn implementation of SVR and my input for the train set is a sparse matrix. But, for my surprise, ...
3
votes
1answer
70 views

Understand the reasons of using Kernel method in SVM

I understand that one can use kernel functions (i.e. radial kernel) to create non-linear decision boundary. However, there is something with my logic and I am sure there is something that I clearly ...
0
votes
1answer
24 views

Local log-likelihood for multiclass linear regression model

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
0
votes
0answers
13 views

What are “second-order dependencies” and “higher order dependencies” (statistics)?

I am reading some materials about PCA and kernel PCA. And it comes to these two notions: "second-order dependencies" and "higher order dependencies". I couldn't find any clear explanation of them. ...
1
vote
1answer
86 views

Kernel density estimator function explanation needed

I'm studying about kernel density estimation and from wikipedia I get this formula: $$\hat{f}_h(x, h) = \frac{1}{n}\sum^{n}_{i=1}K_h(x-x_i) = \frac{1}{nh}\sum_{i=1}^nK(\frac{x-x_i}{h}).$$ I think ...
0
votes
1answer
54 views

Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
0
votes
1answer
60 views

Accuracy changes with permutation of input data in Libsvm with precomputed kernel?

I'm doing quite simple SVM classification at the moment. I use a precomputed kernel in LibSVM with RBF and DTW. When I compute the similarity (kernel-) matrix, everything seems to work very fine ... ...
1
vote
1answer
44 views

Efficient evaluation of multidimensional kernel density estimate

I've seen a reasonable amount of literature about how to choose kernels and bandwidths when computing a kernel density estimate, but I am currently interested in how to improve the time it takes to ...
0
votes
0answers
58 views

What's wrong with my Kernel algorithm (Kernel SVD)?

I have a user-item matrix $A$ as data input, which is a sparse matrix containing a large number of missing values (as zeros). Each row is a user, and each column is an item. Generally, I am conducting ...
1
vote
0answers
22 views

Regression of family of marginal density functions

From a large sample of triples $(X, Y, U)$ I need to estimate a function $(x, y, u) \mapsto f(x, y, u)$, such that for each fixed $x, y$, the function $u \mapsto f(x, y, u)$ is a density function; ...
2
votes
0answers
63 views

Prove that this exponential kernel is positive definite

Let $x,y\in R^d$ and $d:R^d\times R^d \rightarrow R$ a metric on $R^d$ be given. The exponential kernel is defined by: $k(x,x')=e^{−αd(x,x')}$ where $α>0$. The kernel matrix is defined as the ...
2
votes
0answers
112 views

What are the advantages of kernel PCA over PCA?

I want to implement an algorithm in a paper which uses Kernel SVD to decompose a data matrix. So I have been reading materials about Kernel methods and kernel PCA etc. But it still is very obscure to ...
1
vote
1answer
55 views

Cascade Combination of Kernel Functions

I have a question regarding machine learning and specifically kernel functions. Suppose we have a Kernel function, say $K(x)$, and also another distinct one, say $K'(x)$. I want to know is $K(K'(x))$ ...
1
vote
0answers
25 views

Difference between Primal, Dual and Kernel Ridge Regression

I would like to basically ask what the title says. What is the difference between Primal, Dual and Kernel Ridge Regression? People are using all three, and because of the different notation that ...
1
vote
1answer
56 views

Linear Kernel in Baysian Linear Regression

I came up with http://mlg.eng.cam.ac.uk/duvenaud/cookbook/index.html and it is actually very useful. At some point it says If you use just a linear kernel in a GP, you're simply doing Bayesian ...
1
vote
0answers
41 views

Derive squared exponential covariance function

In Gaussian Processes, SVMs, kernels are used (as to my understanding) as similarity measure. However, they have the constraint that any kernel has to be represented as a dot product. i.e. ...
1
vote
2answers
118 views

Prediction with scikit and an precomputed kernel (SVM)

I am kind of a newbie in the MachineLearning area and evaluating some tools etc. to get a feeling for it. For a project I am using a tool that creates a precomputed kernel (gram matrix) and also is ...
2
votes
1answer
57 views

Finding the cluster centers in kernel k-means clustering

I think this is the most easily understood topic in Kernel K Means Clustering. But assuming that I am not an expert in Machine Learning, can someone tell me how does someone calculate Kernel K means ...
0
votes
0answers
30 views

What's wrong with the intuition that kernel measures similarity between observations?

Near the middle of page 16 of Andrew Ng's notes on SVM, he explained an intuitive view of kernel as measuring similarity between observations, but then added the caveat that there are things wrong ...
1
vote
0answers
57 views

Need a working algorithm to find out optimal kernel bandwidth for density estimation

I am looking for a working algorithm for find out optimal kernel bandwidth for density estimation. I need to write my own program in pascal instead of using R or Matlab. So far all algorithms I ...
0
votes
0answers
23 views

Kernel PCA number of components

When using Kernel PCA for dimensionality reduction is there any simple criterion which can be used to determine the number of components to use? I am using Kernel PCA with linear kernel, which would ...
3
votes
1answer
91 views

SVM basic theory?

I have some questions about SVM: In SVM there is a nonlinear and linear SVM. What is the difference between them? To do classification in SVM, we will find the linearly separable boundary ...
0
votes
0answers
35 views

How to find out optimal KDE bandwidth via Bootstrap Aggregation

I am a programmer and trying to do some data analysis. Since I am very interested in statistics, and I have learned a lot of programming languages, learning how to use professional packages such as ...
6
votes
1answer
67 views

Kernel smoothing for Edgeworth expansion

Suppose I have an estimator which includes an indicator function in the objective function, then the objective function is not smooth. But if I want to approximate the behavior of this estimator in ...
2
votes
1answer
54 views

Calculating agreement between an ordinal and continuous scale

I'm having a bit of trouble wrapping my head around something. I have a data set which contains two columns that essentially attempt to measure the same thing, one on a 1-50 continuous scale and ...
0
votes
0answers
23 views

What is the difference between the metric window width and Nearest-neighbor's window in Kernel Smoothing methods?

I'm learning Kernel smoothing methods. I didn't really get the difference between the metric window width and Nearest-neighbor's window. For me both seem the same. Can anybody explain it to me? for ...
0
votes
0answers
36 views

how to prove this kernel function is positive semidefinite

How to prove $k(x_i,x_j)=e^{-(LR(x_i-x_j))^TLR(x_i-x_j)}$ is a valid kernel function or positive semi definite? $x=(\mu,\lambda)^T$ and R is a 2x2 rotation matrix, L is a 2x2 diagonal scaling matrix ...
0
votes
1answer
74 views

Binary classification using radial basis kernel SVM with a single feature

Is there any interpretation (graphical or otherwise) of a radial basis kernel SVM being trained with a single feature? I can visualize the effect in 2 dimensions (the result being a separation ...
1
vote
0answers
64 views

Kernel estimation of hazard or density function

I am working with R package "muhaz" to get the kernel estimate of the hazard function. If we have the following example which is the sample of uncensored data: ...
1
vote
0answers
27 views

kernel for a (semi-) metric space

Let's say I have a metric space $(\mathcal{X}, d)$. Is there any kernel function that I can use with SVM? If we change the RBF kernel a little bit, we have $k(x,y) = e^{-d(x,y)^2}$. Is this a valid ...
3
votes
0answers
30 views

Optimal Kernel Size for Lanczos resampling

The kernel for Lanczos Resampling is defined by $$K(u) = \frac{a\text{sin}(\pi u)\text{sin}(\pi u/a)}{\pi^2u^2}.$$ How does one go about finding a value of $a$ to minimize the mean squared error, ...
0
votes
1answer
51 views

How does a linear SVM work? [duplicate]

I have a 2-class problem involving many features. Does a linear support vector machine (SVM) classifier only take into account the values of these features and nothing more? Does it see the ...
6
votes
1answer
225 views

How to draw random samples from a non-parametric estimated distribution?

I have a sample of 100 points which are continuous and one-dimensional. I estimated its non-parametric density using kernel methods. How can I draw random samples from this estimated distribution?
8
votes
3answers
233 views

Where is density estimation useful?

After going through some slightly terse mathematics, I think I have a slight intuition of kernel density estimation. But I am also aware that estimating multivariate density for more than three ...
1
vote
1answer
55 views

Integral of a function with uniform kernel

I am trying to understand question 9-1 on p.334 in Cameron & Trivedi (link) where I have to calculate the bias of a kernel density estimate at $x=1$ and $n=100$, where we assume that the ...
0
votes
0answers
16 views

scikit-learn SkewedChi2Sampler - meaning of skewedness parameter

I am trying to understand the meaning of the "skewedness" parameter for scikit-learn's SkewedChi2Sampler and figure out how this value affects the output of the sampler. I have looked at the docs ...
0
votes
1answer
44 views

SVM cost, kernel and dimension

Why is it SVM computation cost does not depend on kernel value, dimensions (when separating hyperplane )? Is it because all it does is just classifying and not much calculation involved?
11
votes
1answer
357 views

Ratio of probabilities vs ratio of PDFs

I'm using Bayes to solve a clustering problem. After doing some calculations I end up with the need to obtain the ratio of two probabilities: $$P(A)/P(B)$$ to be able to obtain $P(H|D)$. These ...
2
votes
1answer
91 views

SVM - Infinite dimensional feature space

What is the intuition behind the fact that an SVM with a Gaussian Kernel has infinite dimensional feature space?