Skip to main content
26 votes
Accepted

What does the cost (C) parameter mean in SVM?

I'm trying to give a simple and easy-to-understand answer. A complete answer would likely need to cover everything from the purpose behind SVMs to the finer details of loss and support vectors. If you ...
geekoverdose's user avatar
  • 3,901
9 votes
Accepted

What is the parameter nu in oneClass svm?

The goal of $\nu$ is similar to that of $C$. The parameter $\nu$ is there to finetune the trade-off between overfitting and generalization. The problem with $C$ is that it is positive and unbounded ...
Satwik Bhattamishra's user avatar
8 votes
Accepted

SVM: Why does the number of support vectors decrease when C is increased?

In short, C is the penalty on the slack variables, which measure the degree to which the margin constraints are violated. A training pattern violates the margin constraint if the kernel expansion (i....
Dikran Marsupial's user avatar
8 votes

Is there a way to determine the important features (weight) for an SVM that uses an RBF kernel?

Unfortunately not. Although SVMs are often interpreted as transforming your features into a high-dimensional space and fitting a linear classifier in the new space, the transformation is implicit and ...
mary's user avatar
  • 151
7 votes

What is the influence of C in SVMs with linear kernel?

Most of the answers above are quite good, but let me clarify something for someone like me who had to spent 3 days on understanding the role Parameter C in SVM because of diffrent sources. In book ...
Aakash's user avatar
  • 71
7 votes
Accepted

What is the parameter sigma in svm?

The gamma parameter in the RBF kernel determines the reach of a single training instance. If the value of Gamma is low, then every training instance will have a far reach. Conversely, high values of ...
Ch4mb3rs's user avatar
  • 146
6 votes

What is the influence of C in SVMs with linear kernel?

The answers above are excellent. After carefully reading your questions, I found there are 2 important facts we might overlooked. You are using linear kernel Your training data is linearly separable, ...
luz's user avatar
  • 61
4 votes
Accepted

One-class SVM vs. OneVsRestClassifier for multi-label text classification task

Yes, I've done this very same task. The OneVsRestClassifier in scikit-learn can be used for multi-class or multi-label classification. (when used in multi-label ...
Felipe's user avatar
  • 1,014
4 votes

Support vector regression (LIBSVM) returns out of range outputs when I use out-of-sample data to predict one step ahead (MATLAB)?

If you don't explicitly model such constraints, there is no way for the SVM to adhere to them. Without adding constraints, the outputs of SVM regression can be any real number. The most ...
Marc Claesen's user avatar
  • 18.6k
3 votes

What does the number 'Kernel Option' refer to in SVM?

In LIBSVM, you can use different kernel types by changing the numerical value of the -t input. You can also set several parameter values depending on which kernel ...
Jeffrey Girard's user avatar
3 votes

One-vs-many/One-vs-all - what value to use as probability?

What you want to do is probability estimation through pairwise coupling. Check the paper by Zadrozny B. (2001). Here's the Abstract: This paper presents a method for obtaining class membership ...
Firebug's user avatar
  • 19.3k
3 votes
Accepted

Does LibSVM use Platt Scaling?

Yes, libSVM uses Platt Scaling in its output. Although from the source code, it appears that libSVM authors implemented a modified version of Platt's method in which the maximum likelihood function is ...
Sandeep S. Sandhu's user avatar
3 votes

What is the influence of C in SVMs with linear kernel?

C Parameter is used for controlling the outliers — low C implies we are allowing more outliers, high C implies we are allowing fewer outliers.
H. Irshad 's user avatar
3 votes

Why all "nBSV"s are zero in LIBSVM classification outputs?

It means that your training loss is 0 (i.e., your data is separable).
MotiNK's user avatar
  • 1,499
3 votes
Accepted

Can someone explain the RBF Kernel to me?

The kernel function is a measure of similarity between two sets of features. So in this case, $x'$ and $x$ will both be $5\times 1$ feature vectors (not necessarily the same). $K(x,x')$ is a scalar ...
combo's user avatar
  • 1,257
3 votes
Accepted

What Are Shrinking Heuristics

This shrinking method is explained in section 5.1 of the official documentation [1]. You can find the pdf here. [1]C.-C. Chang ...
horaceT's user avatar
  • 3,352
3 votes
Accepted

Lasso regression / SVM convergence CPU -> GPU

Some models are known not to scale well with data (e.g. SVM, Gaussian processes). You can check this by comparing their big $O$ time complexity and memory usage. GPUs don't magically make arbitrary ...
Tim's user avatar
  • 139k
2 votes
Accepted

TOO low estimated SVM probability for most of the negative test examples?

The first thing you want to do is look at the outputs of your trained csvm (not the posterior probabilities!). What is happening is that the fitSVMPosterior tries to fit a sigmoid through the scores / ...
Tom's user avatar
  • 1,062
2 votes
Accepted

Decreasing the number of negative examples with respect to the positive examples produces good prediction with SVM

This is an anomaly caused by your use of a discontinuous improper accuracy scoring rule. Use an efficient non-arbitrary scoring rule that is designed for this case, related to log-likelihood or the ...
Frank Harrell's user avatar
2 votes
Accepted

Is there any difference between the result for dual and primal of SVM?

At worst, the resulting models should still be approximately equal since this is a convex optimization problem. Their paper at JMLR1 mentions "LIBLINEAR implements a trust region Newton method" for ...
Firebug's user avatar
  • 19.3k
2 votes
Accepted

Do SVM suffer from imbalanced class sizes?

By default, SVM is susceptible to class imbalance. In many implementations, the misclassification penalties can indeed be reweighted to account for an unequal number of samples, but this usually has ...
Trisoloriansunscreen's user avatar
2 votes
Accepted

One-to-rest Weight vectors in one-to-one multiclass SVC

If you do one-to-one classification mode, there is no single weight vector; there are $\binom{m}{2}$ if you have $m$ classes. You can reconstruct each of those vectors from the appropriate entries of <...
Danica's user avatar
  • 25k
2 votes

How to apply SVM model to new data using libsvm in MATLAB?

From the libsvm readme: If labels of test data are unknown, simply use any random values. So just use a vector of ones or zeroes or whatever you are using for labels. You won't be able to trust ...
Jeffrey Girard's user avatar
2 votes

(SVM) Difference between linear kernel and polynomial kernel of degree 1?

Depending on your SVM implementation, there may be a difference. Compared to the linear kernel, the polynomial kernel has an additional parameter $c$ (and $d$ of course): $K(x,y) = (x^\mathsf{T} y + ...
Pablo Rivas's user avatar
2 votes

What is the parameter sigma in svm?

I’m here to understand it too, but I’ll post what my book says about this: The 𝛾 parameter, which we set to gamma=0.1, can be understood as a cut-off parameter for the Gaussian sphere. If we ...
Mr-Programs's user avatar
2 votes
Accepted

Recover $\rho$ of $\nu$-SVM from e1071 package in R

We can recover this by noting that $0 \leq \alpha_i \leq C = \frac{1}{\rho n}$ (from that same section in the LIBSVM paper, substituing $n$ for $l$). Thus, $\rho = \frac{1}{n \max \alpha_i}$. Note ...
MotiNK's user avatar
  • 1,499
2 votes

Which algorithm is implemented in sklearn's SVM method?

Yes - the implementation there is based on libsvm - which does indeed implement Platt's SMO - you can see the details in this paper.
MotiNK's user avatar
  • 1,499
2 votes

Which algorithm is implemented in sklearn's SVM method?

As you noticed, the documentation says it uses LibSVM, but if in doubt, check the source code: ...
Tim's user avatar
  • 139k
1 vote
Accepted

Odd Behavior in Cutoff of Soft Margin SVM

Basically, for $\alpha_i$ such that $0 < \alpha_i < C$ you should have that it is classified to be on the boundary (as you wrote). For numerical stability reasons, one generally takes an average ...
MotiNK's user avatar
  • 1,499
1 vote

How to train an SVR model?

In your second example you are using only very small values for c in grid search: ((0.000001,0.00001,0.0001)) Low values of c in ...
mzunhammer's user avatar
  • 1,188

Only top scored, non community-wiki answers of a minimum length are eligible