SVM data normalization... what about classifying new (training) data?

Question

I've got a big doubt about SVM classification task (and more in general classification task), about data normalization. Let's suppose I've a SVM trained with normalized data, and new data to classify.

1) How do I normalize new data? Please note that I don't know them when I normalized and trained my SVM.

2) Which is the best/proper normalization method? Min-max or zero-mean+variance?

A possible solution that I thought is: once new data arrives, and as we are working with SVs (that are, part of the training data), we can de-normalize the SVs, re-calculate min-Max/mean-var of the new WHOLE dataset, and normalize the new data and re-normalize the SVs. What about this?

Thanks in advance, Ivano

Answer 1

1. Store the mean and standard deviation of the training dataset features. When the test data is received, normalize each feature by subtracting its corresponding training mean and dividing by the corresponding training standard deviation.

2. Normalizition by min/max is usually a very bad idea since it involves scaling your entire data according to two particular observations. This leads your scaling to be dominated by noise. mean/std is a standard procedure and you can even experiment with more robust measures (e.g. median/MAD)

Why scale/normalize? Because of the way the SVM optimization problem is defined, features with higher variance have greater effect on the margin. Usually this doesn't make sense - we'd like our classifier to be 'unit invariant' (e.g. a classifier that combines patients' weight and height shouldn't be affected by the choice of units - kgs or grams, centimeters or meters).

However, I guess that there might be cases in which all of the features are given in the same units and the differences in their variance indeed reflect differences in importance. In such case I'd try to skip scaling/normalization and see what it does to the performance.