0
votes
0answers
7 views

LIBSVM for pre-computed kernel and zero bias (b values is zero)

I want to do binary classification and I'm using LIBSVM library for that. I have a precomputed Kernel and my bias value (b) is zero. Can I do this in LIBSVM or do I have to use some other library? ...
0
votes
0answers
12 views

Usage of libsvm with RBF kernel and no Offset

I'm using libsvm for the binary classification and using a precomputed Kernel. In my particular problem there is no bias term (it's zero). Is there anyway to adjust the bias term in libsvm (and not ...
0
votes
1answer
16 views

What is the toolkit that implements Cost sensitive Support Vector Machine?

I need implementation of cost sensitive support vector machine. The cost is different for each training example (unlike each class). So problem is to solve $max_\alpha$ $-1/2 \sum_{i,j} ...
0
votes
0answers
8 views

Use cases for P-Kernel for SVMs

I've been reading the book by Cristianini on Kernels (2004) where generative kernels (like p-kernel and fisher-kernel, not to be confused with polynomial kernel!) are described. I am interested in ...
1
vote
1answer
126 views

What are basic differences between Kernel Approaches to Unsupervised and Supervised Machine Learning

I got nice graphical representation of Machine learning for clustering / classification. Source: Kernel Approaches to Unsupervised and Supervised Machine Learning by Sun-Yuan Kung Here are my ...
4
votes
2answers
70 views

When do kernel based method perform better than the regular

I am used with linear models. I can see rising use of kernel based method particularly in machine learning. The following is an example Gaussian kernel using ...
0
votes
1answer
57 views

kernels and similarity (in R)

I am trying fit different kernels to calculate similarity matrix in R. Here is example data - X matrix : ...
1
vote
1answer
59 views

Where can I use kernels other than Gaussian (like Cauchy, laplacian) in kernel methods in machine learning? Or maybe in kernel density estimation?

In few papers I read that - kernel used doesn't really matter for kernel density estimation but bandwidth of the kernel is the most important factor. But I did not see any mathematical explanation to ...
0
votes
0answers
5 views

How accurate sum of kernel function needs to be, so that we can use it in Mean shift algorithm (may be for image segmentation)?

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. It is useful for detecting the modes of this density. This is an iterative ...
0
votes
0answers
10 views

In RVM, is the kernel allowed to depend on the full dataset?

I want to use Relevance Vector Machines and need to define my custom made kernel. I was wondering if it is allowed for the kernel to depend on the full dataset. For exampe, I can calculate a certain ...
3
votes
1answer
95 views

Understand the reasons of using Kernel method in SVM

I understand that one can use kernel functions (i.e. radial kernel) to create non-linear decision boundary. However, there is something with my logic and I am sure there is something that I clearly ...
0
votes
1answer
70 views

Clustering structured data: Assessing the similarity of documents that appear in tree structure

Usually when performing text document clustering, similarities across documents are assessed based on the lexical content of documents. But, in my problem, I wish to consider both the lexical content ...
1
vote
1answer
62 views

Cascade Combination of Kernel Functions

I have a question regarding machine learning and specifically kernel functions. Suppose we have a Kernel function, say $K(x)$, and also another distinct one, say $K'(x)$. I want to know is $K(K'(x))$ ...
3
votes
1answer
108 views

SVM basic theory?

I have some questions about SVM: In SVM there is a nonlinear and linear SVM. What is the difference between them? To do classification in SVM, we will find the linearly separable boundary ...
0
votes
0answers
26 views

What is the difference between the metric window width and Nearest-neighbor's window in Kernel Smoothing methods?

I'm learning Kernel smoothing methods. I didn't really get the difference between the metric window width and Nearest-neighbor's window. For me both seem the same. Can anybody explain it to me? for ...
0
votes
1answer
90 views

Binary classification using radial basis kernel SVM with a single feature

Is there any interpretation (graphical or otherwise) of a radial basis kernel SVM being trained with a single feature? I can visualize the effect in 2 dimensions (the result being a separation ...
1
vote
0answers
28 views

kernel for a (semi-) metric space

Let's say I have a metric space $(\mathcal{X}, d)$. Is there any kernel function that I can use with SVM? If we change the RBF kernel a little bit, we have $k(x,y) = e^{-d(x,y)^2}$. Is this a valid ...
2
votes
2answers
161 views

Poorness of Kernel methods on visual pattern recegnition?

I am currently reading the recent papers mainly written by Y. Bengio [1],[2],[3]. There are very strong claims about poorness of Kernel methods on recognizing handwritings in many general cases but ...
1
vote
2answers
151 views

Applying kernel function to input data before giving it to algorithm

I have gene expression data, I do dimensionality reduction and clustering with self organizing maps, but self organizing maps do not perform well with my data. I want to map my data to feature space ...
3
votes
1answer
349 views

What are the limitations of Kernel methods and when to use kernel methods?

Kernel methods are very effective in many supervised classification tasks. So what are the limitations of kernel methods and when to use kernel methods? Especially in the large scale data era, what ...
10
votes
2answers
1k views

Linear kernel and non-linear kernel for support vector machine?

When using support vector machine, are there any guidelines on choosing linear kernel vs. nonlinear kernel, like RBF? I once heard that non-linear kernel tends not to perform well once the number of ...
2
votes
1answer
153 views

Relationship between the kernel and the value of C in SVM's

How exactly does the value of C relate across different kernels that we can use for SVM's? As in, how does it vary when changing the polynomial degree of a kernel or while using a Gaussian kernel?
0
votes
0answers
148 views

I am still confused with Gaussian kernel in SVM

From the slides http://www.csie.ntu.edu.tw/~cjlin/talks/kuleuven_svm.pdf, $$\min \frac{1}{2}w^Tw $$ subject to $$y_i(w^T\phi(x_i)+b)\ge 1,i=1,\cdots,n$$ I think most people are very familiar with ...
8
votes
2answers
488 views

The Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
6
votes
1answer
1k views

Difference between a SVM and a perceptron

I am a bit confused with the difference between an SVM and a perceptron. Let me try to summarize my understanding here, and please feel free to correct where I am wrong and fill in what I have missed. ...
3
votes
2answers
155 views

Loss for Kernel Ridge Regression

Is $||Y-X\beta||_2^2 + \lambda\beta^T K\beta$ , the standard loss-function in kernel ridge regression, or is it different? Also, is the gaussian kernel a standard choice used for the kernel, in ...
1
vote
2answers
143 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
5
votes
2answers
137 views

Why are these proper kernels and how to deduce that they are?

I am struggling to understand kernels and how to determine whether they are proper or not. For these examples can anyone explain why an example is proper and why another example is not. Given K1 and ...
2
votes
1answer
280 views

Weighted covariance matrix using kernels

I would like to create a weighted covariance matrix (say 5 variables) using 3 different time points where the weights come from a kernel function (can be normal, triangular, etc.) but I'm not ...
4
votes
1answer
243 views

Regarding redundant training data in building SVM-based classifier

To build a SVM-based classifier, I have a training data set consisting of N data points. Some of them are redundant. For instance, there have 50 data points which are exactly the same, and there have ...
3
votes
1answer
650 views

Kernelised k Nearest Neighbour

I'm new to kernels and have hit a snag while trying to kernelise kNN. Preliminaries I'm using a polynomial kernel: $K(\mathbf{x},\mathbf{y}) = (1 + \langle \mathbf{x},\mathbf{y} \rangle)^d$ Your ...
1
vote
1answer
398 views

How do I access or compute the posterior covariance matrix returned by kernlab::gausspr R function?

I am looking to compute the covariance matrix of an inferred Gaussian process in R. Below I outline how I would do this manually, but I realize that the kernlab ...
5
votes
2answers
1k views

Use of the Gamma parameter with support vector machines

When using libsvm, the parameter $\gamma$ is a parameter for the kernel function. Its default value is setup as $\frac{1}{Number Of Features}$ Is there any ...
7
votes
2answers
932 views

How to prove there is no finite-dimensional feature space for Gaussian RBF kernel?

How to prove that for the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ there is no finite-dimensional feature space $H$ such that for some $\Phi: \text{R}^n \to H$ we have ...
2
votes
1answer
128 views

Representation within a RKHS framework

Given a p.s.d kernel $Q$, can minimization/maximization of $Tr(X^TQX)$ over X be represented within a reproducing kernel Hilbert space (RKHS) framework? If there is a primary concern with the trace ...
1
vote
1answer
158 views

Possible reason for failing to build a support vector machine

I was trying to build a classifier for a set of documents using a support vector machine. I choose to build the feature space using term occurrence. While experimenting, I found the following ...
7
votes
1answer
634 views

The relationship between the number of support vectors and the number of features

I ran an SVM against a given data set, and made the following observation: If I change the number of features for building the classifier, the number of resulting support vectors will also be changed. ...
2
votes
1answer
167 views

Does the product of two p.s.d kernel matrices result in a kernel matrix?

In a ML setting, where $a_1,..., a_n$ are a set of training points. A kernel function is a function $κ$ that gives the inner product between two vectors in the feature space: $κ(a_i, a_j ) = ψ(a_i) · ...
7
votes
3answers
1k views

Support vector machine for text classification

I am currently having a data set, class 1 with about 8000 short text files and class 2 with about 3000 short text files. I applied LibSVM and tried a couple of parameter combinations in the ...
2
votes
0answers
302 views

Regarding kernel-based naive Bayesian classifier

Are there any good references for kernel-based Naive Bayesian classifier?
3
votes
2answers
3k views

How to select best parameter for polynomial kernel?

I am using LibSVM library for classification. For my problem I am using polynomial kernel and I need to select best parameters (d = degree of polynomial kernel, and ...
0
votes
4answers
289 views

Kernel Selection

I am not an expert in SVM and kernel, so please excuse me if I ask stupid question. Actually, first I want to know how to analyze a dataset to discover its pattern. And second, how can I select ...
0
votes
2answers
508 views

SVM using RBF and nearest neighbor classification method

SVM using RBF kernel is claimed to be similar (equivalent) to the K nearest neighbor classification method. I am not very clear about the analysis process of building this kind of relationship. Thanks ...
5
votes
2answers
1k views

Kernel logistic regression

I heard Kernel Logistic Regression is a classical combination of kernel methods and Logistic regression, but I cannot find any major reference (book, or paper) on this topic. Can you give me any ...
1
vote
2answers
390 views

Polynomial kernel function

Consider SMV with the polynomial kernel $k(x_1,x_2)=(\langle x_1, x_2\rangle + 1)^d,$ where $d > 1.$ Is it true that if the dataset is separated with a hyperplane then the SVM (with the kernel $k$) ...
1
vote
0answers
272 views

Behavior of a sum of kernel functions

Suppose we have 2 kernel functions $K_1(x,y)$ and $K_2(x,y)$. We know, that the dataset ($(x_1,y_1),\ldots,(x_l,y_l),$ $y_i \in \{-1,1\}$ ) is separated with the first one (that is, there are $w,$ ...
4
votes
1answer
374 views

How to run K-means clustering on data points of varying dimensionality?

I'm trying to aggregate $T$ local image descriptors (i.e. histograms) into a vector, namely, the Fisher Vector as described in this paper by H. Jégou et al., Aggregating local image descriptors into ...
2
votes
3answers
138 views

Multi task learning

I have a dataset where all observations are measured several times and reported outcomes correspond to those measurements. In other words, my set of data points looks like $\{x_i, y_{i_1}, y_{i_2}, ...
1
vote
1answer
505 views

Parameters to change for different kernels for SVM

I am carrying out SVM and was interested in knowing the parameters that could be varied for each kernel. I am using 3 kernels: RBF, linear and polynominal. These are the parameters that i think can ...
7
votes
1answer
123 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...