Interpretating the C value in a linear SVM

Question

My C value is very low (close to 0).

Does this mean that my feature (dimensions) have no real separative (and thus predictive) value? (As the SVM basically chooses to ignore the training data because C is almost zero and thus doesn't penalize misclassifications)?

PS: if relevant to your answer:

• Several Cs (2^-10 to 20^10) were tested on a validation dataset and the model with the best accuracy was chosen to execute on the testing dataset.
• the values of my feature (dimensions) are small (between -1 and +1) because of the nature of the data.
• Their weight (w values) is small too.

Answer 1

No one can definitively tell you what a low C value means without playing and looking at your data as well. A low C value could be caused by:

• A low amount of data relative to the dimension
• Inherent noise in the data set
• Improper weight scaling
• Inherent difficult of the problem
• and more

This doesn't even get into issues about how big the change in accuracy is, whether or not the change in the model's objective function is low relative to the change in accuracy.

If you want an idea of how predictive your features are, why don't you look at the accuracies you can get with those features rather than the hyper parameters that got the best accuracy. If you get 99.99999% accuracy with a model, it doesn't matter what the hyperparameters were - the features are discriminative.