Using KNN for prediction, how should I normalize my data?

Is it better to constrain the data to a range, say [0,1], or to force a mean of 0 and sd of 1? Why? Does the type of input data matter (I'll be using both continuous and categorical variables)?

• A more important consideration might be how to scale each variable. Even if all the variables were continuous, I wouldn't necessarily normalize them all the same way -- if the association with the response variable is stronger for x1 than for x2, I'd want to keep the variance on x1 higher than for x2. For example, scale x1 as normal with mean 0 and variance 4, whereas x2 gets variance 1. Oct 3 '13 at 11:46
• Well, that's a step away from simplicity and toward potential overfitting that I wouldn't want to take.
– ASOF
Oct 3 '13 at 12:26

I think that depends on the data. If you know your feature is bounded, you could scale it to $[0,1]$. If it's binary I guess $\{0,1\}$ is a good choice, perhaps $\{-1,1\}$. Now, if it's unbounded, the standardization to $\text Z$-scores $\overline x = 0$, $\sigma=1$ is a reasonable choice.