Tagged Questions

1
vote
1answer
34 views

How to understand effect of RBF SVM

How can I understand what the RBF Kernel in SVM does? I mean I understand the maths, but is there a way to get a feeling when this kernel will be useful? Would results from kNN be related to SVM/RBF ...
0
votes
1answer
59 views

The Lagrange multipliers of SVM

Actually the solve the SVM is to solve the following Lagrangian Equation: If we don't use kernel function, $\langle x^{(i)},x^{(j)}\rangle$ is just the vector vector inner product. The ...
1
vote
1answer
30 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
1
vote
1answer
109 views

Number of kernel evaluations in SVM training

What is the typical number of kernel evaluations (between two training vectors) performed during a (kernelized) Support Vector Machine (SVM) training? I am asking this question because I need to ...
1
vote
0answers
48 views

How to choose kernel functions for support vector regression

Are there any good resources regarding how to design kernels for regression problems, specifically time-series regression type of problem. I am finding the choice of a kernel for regression extremely ...
1
vote
1answer
112 views

How to determine if the data points are linearly separable from an SVM hyperplane

How to know the data points are linearly separable from an SVM hyperplane? How to get the optimal classifier during iteration process? How to calculate the complexity of the SVM model?
0
votes
2answers
233 views

Which SVM kernel to use for a binary classification problem?

I'm a beginner when it comes to support vector machines. Are there some guidelines that say which kernel (e.g. linear, polynomial) is best suited for a specific problem? In my case, I have to classify ...
0
votes
1answer
65 views

SVM decision function

our decision function e.g. in SVMs for binary classification (where the response is labeld by $y_i \in \{-1,1\}$) has the form: $f(\mathbf{x}) = \text{sgn}(\mathbf{w}^\top \mathbf{x} + b)$ where ...
0
votes
0answers
98 views

SVM with svm and svmpath function

I am trying to compare the R functions svm (library: e1071) and svmpath (library svmpath). ...
2
votes
1answer
89 views

Mixed SVM kernel of RBF and linear

I've read some introduction about different kernels for SVM. It seems RBF is a measure of point distance while the basic kernel (i.e. no kernel) splits the space by hyper-planes. I could imagine that ...
3
votes
1answer
85 views

Regarding redundant training data in building SVM-based classifier

To build a SVM-based classifier, I have a training data set consisting of N data points. Some of them are redundant. For instance, there have 50 data points which are exactly the same, and there have ...
0
votes
1answer
479 views

About SVM cost and gamma parameters tuning

I am using R and e1071 package to tune a C-classification SVM. My question is: regardless of the kernel type (linear, ...
1
vote
4answers
228 views

comparing predictive model with hold out set

In Rapid Miner, I created a predictive model (SVM) with Kernel type = polynomial, c= 10, and obtained 80.77% accuracy using cross validation. When compared to hold out set my accuracy on the test ...
2
votes
0answers
39 views

How can I find the number of support vectors in an SVM, depending on kernel?

Here on slide 3 I see the claim that for linearly separable cases the number of support vectors will be d+1, where d is the dimension we are working on. What if my data is not linearly separable and ...
3
votes
2answers
168 views

Use of the Gamma parameter with support vector machines

When using libsvm, the parameter $\gamma$ is a parameter for the kernel function. Its default value is setup as $\frac{1}{Number Of Features}$ Is there any ...
5
votes
2answers
329 views

How to prove there is no finite-dimensional feature space for Gaussian RBF kernel?

How to prove that for the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ there is no finite-dimensional feature space $H$ such that for some $\Phi: \text{R}^n \to H$ we have ...
3
votes
0answers
48 views

Why we solve the dual problem in SVM? [duplicate]

Possible Duplicate: Why bother with the dual problem when fitting SVM? What advantages do we get from solving in the dual?
6
votes
2answers
353 views

How to prove that the radial basis function is a kernel?

How to prove that the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ is a kernel? As far as I understand, in order to prove this we have to prove either of the following: For ...
1
vote
1answer
99 views

Possible reason for failing to build a support vector machine

I was trying to build a classifier for a set of documents using a support vector machine. I choose to build the feature space using term occurrence. While experimenting, I found the following ...
6
votes
1answer
194 views

The relationship between the number of support vectors and the number of features

I ran an SVM against a given data set, and made the following observation: If I change the number of features for building the classifier, the number of resulting support vectors will also be changed. ...
2
votes
1answer
123 views

Does the product of two p.s.d kernel matrices result in a kernel matrix?

In a ML setting, where $a_1,..., a_n$ are a set of training points. A kernel function is a function $κ$ that gives the inner product between two vectors in the feature space: $κ(a_i, a_j ) = ψ(a_i) · ...
5
votes
3answers
425 views

Support vector machine for text classification

I am currently having a data set, class 1 with about 8000 short text files and class 2 with about 3000 short text files. I applied LibSVM and tried a couple of parameter combinations in the ...
4
votes
2answers
144 views

Does a linear SVM behave in the same way as correlation except with the imposition of a large margin?

I want to understand the relationship between correlation and SVMs. My question is based on initial studies that used correlation as a way to examine distributed processing in the cortex with fMRI. ...
1
vote
2answers
815 views

How to select best parameter for polynomial kernel?

I am using LibSVM library for classification. For my problem I am using polynomial kernel and I need to select best parameters (d=degree of polynomial kernel and C=soft margin constant). The LibSVM ...
0
votes
4answers
224 views

Kernel Selection

I am not an expert in SVM and kernel, so please excuse me if I ask stupid question. Actually, first I want to know how to analyze a dataset to discover its pattern. And second, how can I select ...
0
votes
0answers
197 views

SVM using RBF and nearest neighbor classification method

SVM using RBF kernel is claimed to be similar (equivalent) to the K nearest neighbor classification method. I am not very clear about the analysis process of building this kind of relationship. Thanks ...
2
votes
2answers
574 views

Kernel logistic regression

I heard Kernel Logistic Regression is a classical combination of kernel methods and Logistic regression, but I cannot find any major reference (book, or paper) on this topic. Can you give me any ...
1
vote
2answers
274 views

Polynomial kernel function

Consider SMV with the polynomial kernel $k(x_1,x_2)=(\langle x_1, x_2\rangle + 1)^d,$ where $d > 1.$ Is it true that if the dataset is separated with a hyperplane then the SVM (with the kernel $k$) ...
1
vote
1answer
237 views

Parameters to change for different kernels for SVM

I am carrying out SVM and was interested in knowing the parameters that could be varied for each kernel. I am using 3 kernels: RBF, linear and polynominal. These are the parameters that i think can ...
7
votes
1answer
189 views

What is a kernel and what sets it apart from other functions

There seem to be many machine learning algorithms that rely on kernel functions. SVMs and NNs to name but two. So what is the definition of a kernel function and what are the requirements for it to be ...
-1
votes
1answer
327 views

Issue with R-Kernlab SVM Predict [closed]

I have been trying to build SVM classifier but having trouble with predict. modelrbf<-ksvm(set,y,kernel="rbfdot",type="C-svc") Using automatic sigma ...
3
votes
3answers
870 views

Train a SVM-based classifier while taking into account the weight information

Currently I have a data set which are known to belong to two classes, and would like to build a classifier using SVM. However, there exist different confidence levels for this data set. For example, ...
4
votes
1answer
170 views

Can simple vector distance work as a SVM kernel?

I'm considering using the following simple function as an SVM kernel. It basically computes the distance between the 2 input vectors (norm): $K(\vec{x}_1, \vec{x}_2) = \left\| \vec{x}_1- \vec{x}_2 ...
3
votes
1answer
468 views

How does normalization reduce dimensionality of data?

While reading a SVM tutorial, the author makes the following statement on normalization technique for processing the input data: Normalizing data to unit vectors reduces the dimensionality of the ...
3
votes
1answer
473 views

SVM kernels choose without tests

I am having a big trouble to determine the kernel i should use in a non linear SVM without testing before, I want to know if there is any other ways to determine the best kernel without tests ? Is it ...
4
votes
2answers
170 views

Number of eigenfunctions for kernel

While studying machine learning, I've read the following statement: The kernel $K(x,y)=(x\cdot y+1)^d$ , for $x, y \in \mathbb{R}^p$, has $M={p+d \choose d}$ eigenfunctions that span the space of ...
4
votes
2answers
417 views

Implementing the 'kernel trick' for a support vector machine in R

I've heard a bit about the 'kernel trick' for support vector machines, and I was wondering: How do you identify problems that might benefit from the kernel trick? How to implement it in R? Thank ...
3
votes
1answer
465 views

What is the connection between Kernel Logistic Regression and Smoothing Splines?

Working on probabilistic outputs of kernel methods I found the formulation of the SVM as a Penalized Method using the Binomial Deviance (described for example in "The Elements of Statistical Learning ...
5
votes
2answers
762 views

Which kernel method gives the best probability outputs?

Recently I have used Platt's scaling of SVM-outputs to estimate probabilities of default-events. More direct alternatives seem to be "Kernel logistic Regression" (KLR) and the related "Import Vector ...
3
votes
2answers
541 views

VC dimension of SVM with polynomial kernel in $\mathbb{R^{2}}$

What is the VC dimension of SVM with the polynomial kernel $k(x,x')=(1+<x,x'>_{\mathbb{R^{2}}})^{2}$ for binary classification in $\mathbb{R^{2}}$? It would be equal or more than v iff ...