1
vote
1answer
103 views

What are basic differences between Kernel Approaches to Unsupervised and Supervised Machine Learning

I got nice graphical representation of Machine learning for clustering / classification. Source: Kernel Approaches to Unsupervised and Supervised Machine Learning by Sun-Yuan Kung Here are my ...
4
votes
2answers
45 views

When do kernel based method perform better than the regular

I am used with linear models. I can see rising use of kernel based method particularly in machine learning. The following is an example Gaussian kernel using ...
0
votes
1answer
32 views

kernels and similarity (in R)

I am trying fit different kernels to calculate similarity matrix in R. Here is example data - X matrix : ...
1
vote
1answer
45 views

Where can I use kernels other than Gaussian (like Cauchy, laplacian) in kernel methods in machine learning? Or maybe in kernel density estimation?

In few papers I read that - kernel used doesn't really matter for kernel density estimation but bandwidth of the kernel is the most important factor. But I did not see any mathematical explanation to ...
0
votes
0answers
5 views

How accurate sum of kernel function needs to be, so that we can use it in Mean shift algorithm (may be for image segmentation)?

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. It is useful for detecting the modes of this density. This is an iterative ...
0
votes
0answers
10 views

In RVM, is the kernel allowed to depend on the full dataset?

I want to use Relevance Vector Machines and need to define my custom made kernel. I was wondering if it is allowed for the kernel to depend on the full dataset. For exampe, I can calculate a certain ...
3
votes
1answer
77 views

Understand the reasons of using Kernel method in SVM

I understand that one can use kernel functions (i.e. radial kernel) to create non-linear decision boundary. However, there is something with my logic and I am sure there is something that I clearly ...
0
votes
1answer
60 views

Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
1
vote
1answer
58 views

Cascade Combination of Kernel Functions

I have a question regarding machine learning and specifically kernel functions. Suppose we have a Kernel function, say $K(x)$, and also another distinct one, say $K'(x)$. I want to know is $K(K'(x))$ ...
3
votes
1answer
95 views

SVM basic theory?

I have some questions about SVM: In SVM there is a nonlinear and linear SVM. What is the difference between them? To do classification in SVM, we will find the linearly separable boundary ...
0
votes
0answers
25 views

What is the difference between the metric window width and Nearest-neighbor's window in Kernel Smoothing methods?

I'm learning Kernel smoothing methods. I didn't really get the difference between the metric window width and Nearest-neighbor's window. For me both seem the same. Can anybody explain it to me? for ...
0
votes
1answer
78 views

Binary classification using radial basis kernel SVM with a single feature

Is there any interpretation (graphical or otherwise) of a radial basis kernel SVM being trained with a single feature? I can visualize the effect in 2 dimensions (the result being a separation ...
1
vote
0answers
27 views

kernel for a (semi-) metric space

Let's say I have a metric space $(\mathcal{X}, d)$. Is there any kernel function that I can use with SVM? If we change the RBF kernel a little bit, we have $k(x,y) = e^{-d(x,y)^2}$. Is this a valid ...
2
votes
2answers
155 views

Poorness of Kernel methods on visual pattern recegnition?

I am currently reading the recent papers mainly written by Y. Bengio [1],[2],[3]. There are very strong claims about poorness of Kernel methods on recognizing handwritings in many general cases but ...
0
votes
1answer
124 views

Applying kernel function to input data before giving it to algorithm

I have gene expression data, I do dimensionality reduction and clustering with self organizing maps, but self organizing maps do not perform well with my data. I want to map my data to feature space ...
3
votes
1answer
306 views

What are the limitations of Kernel methods and when to use kernel methods?

Kernel methods are very effective in many supervised classification tasks. So what are the limitations of kernel methods and when to use kernel methods? Especially in the large scale data era, what ...
10
votes
2answers
825 views

Linear kernel and non-linear kernel for support vector machine?

When using support vector machine, are there any guidelines on choosing linear kernel vs. nonlinear kernel, like RBF? I once heard that non-linear kernel tends not to perform well once the number of ...
2
votes
1answer
132 views

Relationship between the kernel and the value of C in SVM's

How exactly does the value of C relate across different kernels that we can use for SVM's? As in, how does it vary when changing the polynomial degree of a kernel or while using a Gaussian kernel?
0
votes
0answers
143 views

I am still confused with Gaussian kernel in SVM

From the slides http://www.csie.ntu.edu.tw/~cjlin/talks/kuleuven_svm.pdf, $$\min \frac{1}{2}w^Tw $$ subject to $$y_i(w^T\phi(x_i)+b)\ge 1,i=1,\cdots,n$$ I think most people are very familiar with ...
8
votes
2answers
417 views

The Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
4
votes
1answer
890 views

Difference between a SVM and a perceptron

I am a bit confused with the difference between an SVM and a perceptron. Let me try to summarize my understanding here, and please feel free to correct where I am wrong and fill in what I have missed. ...
3
votes
2answers
149 views

Loss for Kernel Ridge Regression

Is $||Y-X\beta||_2^2 + \lambda\beta^T K\beta$ , the standard loss-function in kernel ridge regression, or is it different? Also, is the gaussian kernel a standard choice used for the kernel, in ...
1
vote
2answers
135 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
5
votes
2answers
134 views

Why are these proper kernels and how to deduce that they are?

I am struggling to understand kernels and how to determine whether they are proper or not. For these examples can anyone explain why an example is proper and why another example is not. Given K1 and ...
2
votes
1answer
254 views

Weighted covariance matrix using kernels

I would like to create a weighted covariance matrix (say 5 variables) using 3 different time points where the weights come from a kernel function (can be normal, triangular, etc.) but I'm not ...
4
votes
1answer
220 views

Regarding redundant training data in building SVM-based classifier

To build a SVM-based classifier, I have a training data set consisting of N data points. Some of them are redundant. For instance, there have 50 data points which are exactly the same, and there have ...
3
votes
1answer
589 views

Kernelised k Nearest Neighbour

I'm new to kernels and have hit a snag while trying to kernelise kNN. Preliminaries I'm using a polynomial kernel: $K(\mathbf{x},\mathbf{y}) = (1 + \langle \mathbf{x},\mathbf{y} \rangle)^d$ Your ...
1
vote
1answer
356 views

How do I access or compute the posterior covariance matrix returned by kernlab::gausspr R function?

I am looking to compute the covariance matrix of an inferred Gaussian process in R. Below I outline how I would do this manually, but I realize that the kernlab ...
5
votes
2answers
1k views

Use of the Gamma parameter with support vector machines

When using libsvm, the parameter $\gamma$ is a parameter for the kernel function. Its default value is setup as $\frac{1}{Number Of Features}$ Is there any ...
7
votes
2answers
878 views

How to prove there is no finite-dimensional feature space for Gaussian RBF kernel?

How to prove that for the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ there is no finite-dimensional feature space $H$ such that for some $\Phi: \text{R}^n \to H$ we have ...
2
votes
1answer
126 views

Representation within a RKHS framework

Given a p.s.d kernel $Q$, can minimization/maximization of $Tr(X^TQX)$ over X be represented within a reproducing kernel Hilbert space (RKHS) framework? If there is a primary concern with the trace ...
1
vote
1answer
155 views

Possible reason for failing to build a support vector machine

I was trying to build a classifier for a set of documents using a support vector machine. I choose to build the feature space using term occurrence. While experimenting, I found the following ...
7
votes
1answer
572 views

The relationship between the number of support vectors and the number of features

I ran an SVM against a given data set, and made the following observation: If I change the number of features for building the classifier, the number of resulting support vectors will also be changed. ...
2
votes
1answer
163 views

Does the product of two p.s.d kernel matrices result in a kernel matrix?

In a ML setting, where $a_1,..., a_n$ are a set of training points. A kernel function is a function $κ$ that gives the inner product between two vectors in the feature space: $κ(a_i, a_j ) = ψ(a_i) · ...
7
votes
3answers
1k views

Support vector machine for text classification

I am currently having a data set, class 1 with about 8000 short text files and class 2 with about 3000 short text files. I applied LibSVM and tried a couple of parameter combinations in the ...
2
votes
0answers
285 views

Regarding kernel-based naive Bayesian classifier

Are there any good references for kernel-based Naive Bayesian classifier?
3
votes
2answers
2k views

How to select best parameter for polynomial kernel?

I am using LibSVM library for classification. For my problem I am using polynomial kernel and I need to select best parameters (d = degree of polynomial kernel, and ...
0
votes
4answers
286 views

Kernel Selection

I am not an expert in SVM and kernel, so please excuse me if I ask stupid question. Actually, first I want to know how to analyze a dataset to discover its pattern. And second, how can I select ...
0
votes
2answers
435 views

SVM using RBF and nearest neighbor classification method

SVM using RBF kernel is claimed to be similar (equivalent) to the K nearest neighbor classification method. I am not very clear about the analysis process of building this kind of relationship. Thanks ...
5
votes
2answers
1k views

Kernel logistic regression

I heard Kernel Logistic Regression is a classical combination of kernel methods and Logistic regression, but I cannot find any major reference (book, or paper) on this topic. Can you give me any ...
1
vote
2answers
377 views

Polynomial kernel function

Consider SMV with the polynomial kernel $k(x_1,x_2)=(\langle x_1, x_2\rangle + 1)^d,$ where $d > 1.$ Is it true that if the dataset is separated with a hyperplane then the SVM (with the kernel $k$) ...
1
vote
0answers
257 views

Behavior of a sum of kernel functions

Suppose we have 2 kernel functions $K_1(x,y)$ and $K_2(x,y)$. We know, that the dataset ($(x_1,y_1),\ldots,(x_l,y_l),$ $y_i \in \{-1,1\}$ ) is separated with the first one (that is, there are $w,$ ...
4
votes
1answer
346 views

How to run K-means clustering on data points of varying dimensionality?

I'm trying to aggregate $T$ local image descriptors (i.e. histograms) into a vector, namely, the Fisher Vector as described in this paper by H. Jégou et al., Aggregating local image descriptors into ...
2
votes
3answers
134 views

Multi task learning

I have a dataset where all observations are measured several times and reported outcomes correspond to those measurements. In other words, my set of data points looks like $\{x_i, y_{i_1}, y_{i_2}, ...
1
vote
1answer
462 views

Parameters to change for different kernels for SVM

I am carrying out SVM and was interested in knowing the parameters that could be varied for each kernel. I am using 3 kernels: RBF, linear and polynominal. These are the parameters that i think can ...
7
votes
1answer
121 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...
3
votes
3answers
2k views

Train a SVM-based classifier while taking into account the weight information

Currently I have a data set which are known to belong to two classes, and would like to build a classifier using SVM. However, there exist different confidence levels for this data set. For example, ...
3
votes
1answer
217 views

Weights of radial basis function networks

If I use radial basis function networks (RBFNs) for probability estimation by plugging the output of the RBFNs into the Logistic function are weights between 0 and 1 sufficient?
3
votes
1answer
558 views

How does normalization reduce dimensionality of data?

While reading a SVM tutorial, the author makes the following statement on normalization technique for processing the input data: Normalizing data to unit vectors reduces the dimensionality of the ...
2
votes
2answers
543 views

Which kernel function for Watson Nadaraya classifier?

I am trying to implement a Watson Nadaraya classifier. There is one thing I didn't understand from the equation: $${F}(x)=\frac{\sum_{i=1}^n K_h(x-X_i) Y_i}{\sum_{i=1}^nK_h(x-X_i)}$$ What should I ...