How many times should I scale my data in machine learning?

Question

I am doing a machine learning project where I need to do

1. PCA
2. then K-means clustering,
3. then One class SVM

It seems all those procedure requires data scaling. Should I (A) scale my data just before PCA, or should I (B) scale my data every time I encounter a new procedure?

(B) will be like scaling the data before PCA, then scaling the data again before K-means,then scaling the data again before One class SVM. I am feeling (B) may lose the original information because I have been scaling it too many times.

Answer 1

In general the answer would be yes - many methods require (or behave better) once data is scaled. This will lose some original meaning, but this is the whole point of normalization - you remove some relations from data to reduce bias coming from the representation. This is just an idea of "assume as low as you can" - this does not mean that lack of normalization is always worse, it just means that "statisticaly speaking" there is a greater chance of good results after normalization, nothing more.

Answer 2

From what I can tell, you are going to feed the output from PCA into k-means clustering and then feed the output of that into SVM. If that's correct, it seems to me you'll only need to scale the data once, when it goes into PCA. The output from PCA will then be scaled (since the outputs are the principal components of the scaled data) and the output of k-means will also have the same scaling (since it is finding group means of scaled data). You can verify that the PCA and k-means outputs are scaled by, let's say, running on a relatively small subset of the data.

Of course, it goes without saying that it helps to understand what each model is doing, instead of treating each one as an incomprehensible black box. Take a look at Brian Ripley's book, "Pattern Recognition and Neural Networks" and/or Hastie, Tibshirani, and Friedman, "Introduction to Statistical Learning."