Tagged Questions

1
vote
1answer
30 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
0
votes
0answers
23 views

intersection kernel and distances between two histograms

intersection kernel can be given as $\sum_i min(x_i, y_i)$ . where x and y are histograms. If two histograms are compeletely different the distance will be low. If two histograms are similar what ...
-1
votes
0answers
23 views

kernels distances gram matrix classification

Could you please explain some thing about kernels? As I understand it is technique to map the feature space into a high dimensional feature space where we could separate two classes by a linear ...
4
votes
2answers
68 views

Why are these proper kernels and how to deduce that they are?

I am struggling to understand kernels and how to determine whether they are proper or not. For these examples can anyone explain why an example is proper and why another example is not. Given K1 and ...
1
vote
0answers
83 views

Weighted covariance matrix using kernels

I would like to create a weighted covariance matrix (say 5 variables) using 3 different time points where the weights come from a kernel function (can be normal, triangular, etc) but I'm not ...
3
votes
1answer
84 views

Regarding redundant training data in building SVM-based classifier

To build a SVM-based classifier, I have a training data set consisting of N data points. Some of them are redundant. For instance, there have 50 data points which are exactly the same, and there have ...
1
vote
1answer
163 views

Kernelised k Nearest Neighbour

I'm new to kernels and have hit a snag while trying to kernelise kNN. Preliminaries I'm using a polynomial kernel: $K(\mathbf{x},\mathbf{y}) = (1 + \langle \mathbf{x},\mathbf{y} \rangle)^d$ Your ...
0
votes
1answer
101 views

How do I access or compute the posterior covariance matrix returned by kernlab::gausspr R function?

I am looking to compute the covariance matrix of an inferred Gaussian process in R. Below I outline how I would do this manually, but I realize that the kernlab ...
3
votes
2answers
165 views

Use of the Gamma parameter with support vector machines

When using libsvm, the parameter $\gamma$ is a parameter for the kernel function. Its default value is setup as $\frac{1}{Number Of Features}$ Is there any ...
5
votes
2answers
321 views

How to prove there is no finite-dimensional feature space for Gaussian RBF kernel?

How to prove that for the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ there is no finite-dimensional feature space $H$ such that for some $\Phi: \text{R}^n \to H$ we have ...
2
votes
0answers
93 views

Representation within a RKHS framework

Given a p.s.d kernel $Q$, can minimization/maximization of $Tr(X^TQX)$ over X be represented within a reproducing kernel Hilbert space (RKHS) framework? If there is a primary concern with the trace ...
1
vote
1answer
99 views

Possible reason for failing to build a support vector machine

I was trying to build a classifier for a set of documents using a support vector machine. I choose to build the feature space using term occurrence. While experimenting, I found the following ...
6
votes
1answer
190 views

The relationship between the number of support vectors and the number of features

I ran an SVM against a given data set, and made the following observation: If I change the number of features for building the classifier, the number of resulting support vectors will also be changed. ...
2
votes
1answer
121 views

Does the product of two p.s.d kernel matrices result in a kernel matrix?

In a ML setting, where $a_1,..., a_n$ are a set of training points. A kernel function is a function $κ$ that gives the inner product between two vectors in the feature space: $κ(a_i, a_j ) = ψ(a_i) · ...
5
votes
3answers
419 views

Support vector machine for text classification

I am currently having a data set, class 1 with about 8000 short text files and class 2 with about 3000 short text files. I applied LibSVM and tried a couple of parameter combinations in the ...
2
votes
0answers
136 views

Regarding kernel-based naive Bayesian classifier

Are there any good references for kernel-based Naive Bayesian classifier?
1
vote
2answers
794 views

How to select best parameter for polynomial kernel?

I am using LibSVM library for classification. For my problem I am using polynomial kernel and I need to select best parameters (d=degree of polynomial kernel and C=soft margin constant). The LibSVM ...
0
votes
4answers
220 views

Kernel Selection

I am not an expert in SVM and kernel, so please excuse me if I ask stupid question. Actually, first I want to know how to analyze a dataset to discover its pattern. And second, how can I select ...
0
votes
0answers
196 views

SVM using RBF and nearest neighbor classification method

SVM using RBF kernel is claimed to be similar (equivalent) to the K nearest neighbor classification method. I am not very clear about the analysis process of building this kind of relationship. Thanks ...
2
votes
2answers
567 views

Kernel logistic regression

I heard Kernel Logistic Regression is a classical combination of kernel methods and Logistic regression, but I cannot find any major reference (book, or paper) on this topic. Can you give me any ...
1
vote
2answers
271 views

Polynomial kernel function

Consider SMV with the polynomial kernel $k(x_1,x_2)=(\langle x_1, x_2\rangle + 1)^d,$ where $d > 1.$ Is it true that if the dataset is separated with a hyperplane then the SVM (with the kernel $k$) ...
1
vote
0answers
170 views

Behavior of a sum of kernel functions

Suppose we have 2 kernel functions $K_1(x,y)$ and $K_2(x,y)$. We know, that the dataset ($(x_1,y_1),\ldots,(x_l,y_l),$ $y_i \in \{-1,1\}$ ) is separated with the first one (that is, there are $w,$ ...
4
votes
1answer
243 views

How to run K-means clustering on data points of varying dimensionality?

I'm trying to aggregate $T$ local image descriptors (i.e. histograms) into a vector, namely, the Fisher Vector as described in this paper by H. Jégou et al., Aggregating local image descriptors into ...
2
votes
3answers
115 views

Multi task learning

I have a dataset where all observations are measured several times and reported outcomes correspond to those measurements. In other words, my set of data points looks like $\{x_i, y_{i_1}, y_{i_2}, ...
1
vote
1answer
235 views

Parameters to change for different kernels for SVM

I am carrying out SVM and was interested in knowing the parameters that could be varied for each kernel. I am using 3 kernels: RBF, linear and polynominal. These are the parameters that i think can ...
6
votes
1answer
68 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...
3
votes
3answers
838 views

Train a SVM-based classifier while taking into account the weight information

Currently I have a data set which are known to belong to two classes, and would like to build a classifier using SVM. However, there exist different confidence levels for this data set. For example, ...
2
votes
1answer
180 views

Weights of radial basis function networks

If I use radial basis function networks (RBFNs) for probability estimation by plugging the output of the RBFNs into the Logistic function are weights between 0 and 1 sufficient?
3
votes
1answer
464 views

How does normalization reduce dimensionality of data?

While reading a SVM tutorial, the author makes the following statement on normalization technique for processing the input data: Normalizing data to unit vectors reduces the dimensionality of the ...
2
votes
2answers
241 views

Which kernel function for Watson Nadaraya classifier?

I am trying to implement a Watson Nadaraya classifier. There is one thing I didn't understand from the equation: $${F}(x)=\frac{\sum_{i=1}^n K_h(x-X_i) Y_i}{\sum_{i=1}^nK_h(x-X_i)}$$ What should I ...
0
votes
2answers
91 views

Viewing kernel regression in a Bayesian framework

If one wanted to use Kernel Regression in a Bayesian Framework, any ideas on how one would go about it? Kernel Regression
4
votes
2answers
170 views

Number of eigenfunctions for kernel

While studying machine learning, I've read the following statement: The kernel $K(x,y)=(x\cdot y+1)^d$ , for $x, y \in \mathbb{R}^p$, has $M={p+d \choose d}$ eigenfunctions that span the space of ...
4
votes
2answers
217 views

Learning a univariate transform (kernel?) for novelty detection

I have 150 observations, 500 features, and I am interested in novelty detection (outlier detection): given a new observation (let's say 'patient') I want to know if it is different from the previous ...
4
votes
2answers
417 views

Implementing the 'kernel trick' for a support vector machine in R

I've heard a bit about the 'kernel trick' for support vector machines, and I was wondering: How do you identify problems that might benefit from the kernel trick? How to implement it in R? Thank ...
3
votes
1answer
458 views

What is the connection between Kernel Logistic Regression and Smoothing Splines?

Working on probabilistic outputs of kernel methods I found the formulation of the SVM as a Penalized Method using the Binomial Deviance (described for example in "The Elements of Statistical Learning ...
9
votes
2answers
558 views

Applying the “kernel trick” to linear methods?

The kernel trick is used in several machine learning models (e.g. SVM). It was first introduced in the "Theoretical foundations of the potential function method in pattern recognition learning" paper ...