5
$\begingroup$

I've got a big doubt about SVM classification task (and more in general classification task), about data normalization. Let's suppose I've a SVM trained with normalized data, and new data to classify.

1) How do I normalize new data? Please note that I don't know them when I normalized and trained my SVM.

2) Which is the best/proper normalization method? Min-max or zero-mean+variance?

A possible solution that I thought is: once new data arrives, and as we are working with SVs (that are, part of the training data), we can de-normalize the SVs, re-calculate min-Max/mean-var of the new WHOLE dataset, and normalize the new data and re-normalize the SVs. What about this?

Thanks in advance, Ivano

$\endgroup$
  • 1
    $\begingroup$ A comment: and what about NOT scaling/normalizing? $\endgroup$ – the_WaterKey Apr 16 '14 at 16:58
8
$\begingroup$
  1. Store the mean and standard deviation of the training dataset features. When the test data is received, normalize each feature by subtracting its corresponding training mean and dividing by the corresponding training standard deviation.

  2. Normalizition by min/max is usually a very bad idea since it involves scaling your entire data according to two particular observations. This leads your scaling to be dominated by noise. mean/std is a standard procedure and you can even experiment with more robust measures (e.g. median/MAD)

Why scale/normalize? Because of the way the SVM optimization problem is defined, features with higher variance have greater effect on the margin. Usually this doesn't make sense - we'd like our classifier to be 'unit invariant' (e.g. a classifier that combines patients' weight and height shouldn't be affected by the choice of units - kgs or grams, centimeters or meters).

However, I guess that there might be cases in which all of the features are given in the same units and the differences in their variance indeed reflect differences in importance. In such case I'd try to skip scaling/normalization and see what it does to the performance.

$\endgroup$
  • $\begingroup$ +1 I especially appreciate the points made in #2, which are frequently overlooked. $\endgroup$ – whuber Apr 16 '14 at 19:29
  • $\begingroup$ Why not to normalize test set using the mean and std deviation of the test set itself rather than using those metrics from the test set? $\endgroup$ – Anmol Singh Jaggi May 7 '16 at 12:33
  • $\begingroup$ Reasons to prefer normalization by the training data: 1. The training data is usually much larger, allowing better estimates of the mean and std. 2. In real world application, the testing data can be a single observation, so no statistics are available. 3. The test data is unlabeled and not necessary balanced, so normalization might be affected by the true, unknown test labels. Having said all that, there can be a situation where the test data is scaled and/or shifted (e.g. obtained by a different, uncalibrated sensor), and then normalizing it by its own mean and std makes sense. $\endgroup$ – Trisoloriansunscreen May 7 '16 at 18:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.