My C value is very low (close to 0).

Does this mean that my feature (dimensions) have no real separative (and thus predictive) value? (As the SVM basically chooses to ignore the training data because C is almost zero and thus doesn't penalize misclassifications)?

PS: if relevant to your answer:

  • Several Cs (2^-10 to 20^10) were tested on a validation dataset and the model with the best accuracy was chosen to execute on the testing dataset.
  • the values of my feature (dimensions) are small (between -1 and +1) because of the nature of the data.
  • Their weight (w values) is small too.

No one can definitively tell you what a low C value means without playing and looking at your data as well. A low C value could be caused by:

  • A low amount of data relative to the dimension
  • Inherent noise in the data set
  • Improper weight scaling
  • Inherent difficult of the problem
  • and more

This doesn't even get into issues about how big the change in accuracy is, whether or not the change in the model's objective function is low relative to the change in accuracy.

If you want an idea of how predictive your features are, why don't you look at the accuracies you can get with those features rather than the hyper parameters that got the best accuracy. If you get 99.99999% accuracy with a model, it doesn't matter what the hyperparameters were - the features are discriminative.

| cite | improve this answer | |
  • $\begingroup$ Thank you for your answer!I think the first two points are probably the reason. I'll look at the different accuracy percentages for my features. Final question for that: am I right in assuming that in the following picture (matrix), there are 35 features, or is it 1 feature with 35 dimensions: imgur.com/BfnwkZE ). $\endgroup$ – Glenn Jul 20 '14 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.