# Structural risk minimization in SVMs

SVM with gaussian kernel (RBF kernel) have infinite VC-dimension and the VC-dimension for SVM with polynomial kernels is very big too. Thus, I wonder how is possible that SVM have good generalization performances. In SRM (structural risk minimization) as larger is the VC dimension as larger is the risk. I know that SVM algorithm selects the hyperplane with minimum VC dimension (namely with minimum margin) however if the VC-dimension is infinite (like in gaussian kernels) this minimum is infinite and the risk will be likely high. How is it possible?

(the question is more specific than an other already existing)

• You mean maximum margin not minimum margin – MotiNK May 11 '17 at 11:42

The generalization capabilities of SVM arise through the margin, see for example herehttps://www.cs.utah.edu/~piyush/teaching/27-9-print.pdf on page 13 where they use Vapnik's bound on classifiers having a specific margin $\gamma$ (these are actually for classifiers which don't make mistakes on the training data but the point is illustrative). Specifically: $$VC(H_\gamma) \leq \min\left\{D, \left\lceil\frac{4R^2}{\gamma^2}\right\rceil\right\}$$ where $D$ is the dimension of the data, and $||x_i|| \leq R$ for all data (so for RBF, $D=\infty$ and $R=1$)
• Your statement is inaccurate. The VC-dimension for SVM is upper-bounded by the value you give - it is not equal to it for all margins. Yes, with RBF kernel if we have a margin $\gamma \rightarrow 0$ then indeed our bound does become infinite, but if it is lower-bounded for the data, then we have a limit to the VC-dimension. In other words, SVM with RBF kernel CAN have infinite VC-dimension, but by maximizing the margin we choose a classifier belonging to a class of limited VC-dimension (and given the kernel it is indeed the minimizer of the above bound). – MotiNK May 12 '17 at 7:31