SVM Model: What's a healthy number of support vectors?

Question

For a SVM model what is a healthy number of support vectors? or more precisely what's a good ratio of number of support vectors to the total number of training samples, 10%, 20%, 30%, 50% ... 80%? Is there a general consensus on this?

By healthy I mean that the SVM model is a good fit and has good generalization power.

For example, I fit a SVM model with 50 predictors, two response classes and the support ratio is about 25%. I have solid out of sample performance i.e. F1, accuracy etc all scoring higher OS than IS but does this support ratio makes sense or is it too high?

Answer 1

It depends on the data (number of features, specifically; as well as expected level of noise) and on the kernel used, but I typically doubt my SVM model if it uses more than 20% of the data as support vectors. When you use 50% or more, you might as well use an RBF model or $k$-nearest neighbors with a suitable distance metric.

I would aim for 10% or less, really; depending on the problem, it might be a bit higher.

Answer 2

The number of support vectors is largely determined by the number of training errors you have - any point having non-negative loss is a support vector. Thus, the training error is the minimum number. Note for $\nu$-SVM, if $\nu$ is "in-range", then $\nu$ represents a bound on the fraction of SVs. It's important to note that $\nu$ can be too small - this is equivalent to $C=\infty$ or $\lambda=0$ in the more standard SVM formulations.

Additionally, though, in certain contexts (e.g., High Dimension, Low Sample Size) - you may also experience "piling" whereby there are a large number of support vectors on the margins - see here for example.