Do non-linear Support Vector Machines need data to be scaled with zero mean and unit std?

Question

I am trying to understand the difference between standardisation, normalisation and scaling for features. My data roughly follows a bell shaped/Gaussian distribution but the magnitude of their respective scales are enormous to miniscule.

The data ranges from negative to positive as well. Would sklearn.preprocessing.StandardScaler().fit_transform(features) work? Or do I need a different data preprocessing method.

Answer 1

Short answer: yes. Your choice of kernel function may very well be sensitive to feature scaling. Since the SVM generates a "linear" boundary in the kernel space, different scalings of features will affect how that classification occurs. Recall that SVMs classify points based on the value of $sign(\phi(w)^\top \phi(x) + b)$, where $\phi(\cdot)$ is the feature map induced on our data. The feature representation $\phi(x)$ and by extension its prediction can dramatically change given the scaling of the components of $x$.

See this answer for an explanation of this effect on the RBF kernel.

For the polynomial kernel, this is even clearer because the feature mapping it induces is finite and describable. I won't derive it here, but if you'd like to know, pages 14-16 provide pretty good intuition. You will notice that as the components of your data vector changes, so does the kernel -- polynomially.

Basically, what you'll realize from these is that the kernel function is an inner product in the feature space. This means that not only does arbitrarily scaling your data similarly scale the "influence" the data points have; that is, extreme points in one dimension will have a disproportionate impact on the classification since that term in the inner product computation accounts for most of the magnitude of the inner product.