why is rbf kernel svm a non-parametric algorithm?

Question

I was reading up the difference between parametric and non-parametric models on this site: https://sebastianraschka.com/faq/docs/parametric_vs_nonparametric.html

It says that linear SVM is parametric but RBF is non-parametric because "In the RBF kernel SVM, we construct the kernel matrix by computing the pair-wise distances between the training points, which makes it non-parametric."

The linear SVM uses the primal form which is :

whereas kernalized SVMs use only the dual form which is:

From the above equations, I don't see how linear SVM are parametric but Kernalized ones are non- parametric.

Answer 1

A model is a family of distributions or functions indexed by a parameter vector $\theta$. In parametric models, $\theta$ has a fixed, finite dimensionality. In nonparametric models, $\theta$ may be infinite dimensional. In practice, we deal with this infinite dimensional space by working with finite parameters whose number/dimensionality grows with the amount of training data.

This distinction can be seen in the SVM equations you wrote. Parameters of the linear SVM consist of the weight vector $w$ and bias $b$. So, there are $d+1$ parameters for a $d$-dimensional input space. This number is fixed, and does not depend on $n$, the number of training points. So, the linear SVM is parametric.

Parameters of the kernelized SVM include the full training set $\{(x_i, y_i)\}_{i=1}^n$, a weight $\alpha_i$ for each training point, and a bias $w_0$. The kernel function may include additional hyperparameters. Clearly, the number of parameters grows with the number of training points. So, the kernelized SVM is nonparametric.