4
$\begingroup$

As far as I know, the only assumptions of support vector machines are independent and identically distributed data. I am planning to train and run a SVM on a number of variables that aren't naturally on the same scale. To meet the assumption of identically distributed data, I was planning on standardizing the variables; however, I'm not sure if I should do this for the training set and test set individually, or for the overall sample prior to creating the two sets of data.

It seems to me that it would be better to standardize the training set and test set individually, but I have no evidence to back this up and no citation to point to. Does anyone know if this is true? Also, is there a citation on this topic?

$\endgroup$
9
$\begingroup$

The correct procedure is to scale the data separately in the following way:

  1. Divide training and test data.
  2. For the training data, center and scale the data. Retain the values of the centering and scaling.
  3. Using the values from (2), subtract the center from the test data and divide by the scale.

A reference for this in the context of support vector machines can be found here, "A Practical Guide to Support Vector Classification" Chih-Wei Hsu, Chih-Chung Chang, and Chih-Jen Lin Department of Computer Science National Taiwan University.

The reason for this is that we do not want information to spill over between the test and training sets. This is true in general of any ML procedure, not just SVM.

$\endgroup$
5
$\begingroup$

That paper does not say that an SVM assumes independent and identically distributed data, but just that the statistical analysis undertaken in that paper makes that assumption. Moreover, this assumption does not refer to the different variables ("features") but to the different data points in each class. Scaling may improve SVM performance, but identical distributions for different variables is in no way a prerequisite.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.