I'm trying to use a Support Vector Machine for classification using Scikit-Learn while understanding how to tune the hyperparameters.

My original dataset has ~4000 features and ~150 samples. I've tried a few transformation techniques to get the data into ~10000 samples and 5 features. My features range from 0 - 1 and are scaled per sample. My classification tasks has 5 different targets.

There are 3 options for classifiers:

SVC | C-Support Vector Classification.

NuSVC | Nu-Support Vector Classification

LinearSVC | Linear Support Vector Classification

I have been using the SVC algorithm but only because I don't understand what is happening with the NuSVC.

When would you use one over the other (e.g. SVC over NuSVC?)

I also don't understand when a Linear kernel would be desirable. If the data was linearly separable, wouldn't a more simple algorithm be used like a LogisticRegression?

I understand the basis of how a Support Vector Machine works and creating hyperplanes that separate out the classes but there are lots of hyperparameters that are slightly esoteric.

For example, my hyperparameter confusion:

  1. What is shrinking?
  2. How does coef0 affect the kernel?
  3. How does gamma affect the kernel?
  4. If I set kernel="linear" for SVC, would that make it the same as LinearSVC?
  5. What penalty is C referring to? Does it use l2 loss by default? What is the range for C?
  • 1
    $\begingroup$ The question is far too broad. What is the kind of data you are working with? How many features? What type of features? How many classes? How many samples? How many samples per class?.... Please provide details about your concrete problem, so that somebody can give a concrete answer. $\endgroup$ – jpmuc Jul 9 '17 at 18:03
  • $\begingroup$ I didn't originally add the details on my training data to keep it general but I've just added them on. I haven't found a good tutorial or documentation on tuning these hyperparameters. $\endgroup$ – O.rka Jul 9 '17 at 18:13
  • $\begingroup$ 1. regarding shrinking - stats.stackexchange.com/questions/24414/… $\endgroup$ – Nikolas Rieble Jul 10 '17 at 9:12
  • $\begingroup$ 2. and 3, regarding coef0 and gamma - did you see scikit-learn.org/stable/modules/svm.html#svm-kernels $\endgroup$ – Nikolas Rieble Jul 10 '17 at 9:12
  • $\begingroup$ 4. stackoverflow.com/questions/35076586/… $\endgroup$ – Nikolas Rieble Jul 10 '17 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.