2
$\begingroup$

For a SVM model what is a healthy number of support vectors? or more precisely what's a good ratio of number of support vectors to the total number of training samples, 10%, 20%, 30%, 50% ... 80%? Is there a general consensus on this?

By healthy I mean that the SVM model is a good fit and has good generalization power.

For example, I fit a SVM model with 50 predictors, two response classes and the support ratio is about 25%. I have solid out of sample performance i.e. F1, accuracy etc all scoring higher OS than IS but does this support ratio makes sense or is it too high?

$\endgroup$

2 Answers 2

1
$\begingroup$

It depends on the data (number of features, specifically; as well as expected level of noise) and on the kernel used, but I typically doubt my SVM model if it uses more than 20% of the data as support vectors. When you use 50% or more, you might as well use an RBF model or $k$-nearest neighbors with a suitable distance metric.

I would aim for 10% or less, really; depending on the problem, it might be a bit higher.

$\endgroup$
2
  • $\begingroup$ can you comment on the updated question? I am really curious ... $\endgroup$
    – SkyWalker
    Nov 18, 2020 at 16:43
  • 1
    $\begingroup$ With 50 features, that might be a reasonable number. It really depends on how complex the class boundary is. It looks like you've done all the right tests. You can always try different parameter values to reduce the number of support vectors and compare results. $\endgroup$ Nov 18, 2020 at 17:05
1
$\begingroup$

The number of support vectors is largely determined by the number of training errors you have - any point having non-negative loss is a support vector. Thus, the training error is the minimum number. Note for $\nu$-SVM, if $\nu$ is "in-range", then $\nu$ represents a bound on the fraction of SVs. It's important to note that $\nu$ can be too small - this is equivalent to $C=\infty$ or $\lambda=0$ in the more standard SVM formulations.

Additionally, though, in certain contexts (e.g., High Dimension, Low Sample Size) - you may also experience "piling" whereby there are a large number of support vectors on the margins - see here for example.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.