Is it better to constrain the data to a range, say [0,1], or to force a mean of 0 and sd of 1? Why? Does the type of input data matter (I'll be using both continuous and categorical variables)?

  • $\begingroup$ A more important consideration might be how to scale each variable. Even if all the variables were continuous, I wouldn't necessarily normalize them all the same way -- if the association with the response variable is stronger for x1 than for x2, I'd want to keep the variance on x1 higher than for x2. For example, scale x1 as normal with mean 0 and variance 4, whereas x2 gets variance 1. $\endgroup$
    – zkurtz
    Oct 3 '13 at 11:46
  • $\begingroup$ Well, that's a step away from simplicity and toward potential overfitting that I wouldn't want to take. $\endgroup$
    – ASOF
    Oct 3 '13 at 12:26

I think that depends on the data. If you know your feature is bounded, you could scale it to $[0,1]$. If it's binary I guess $\{0,1\}$ is a good choice, perhaps $\{-1,1\}$. Now, if it's unbounded, the standardization to $\text Z$-scores $\overline x = 0$, $\sigma=1$ is a reasonable choice.


Similar to K-means, KNN uses distance measure. Therefore

  • It is better to normalize features. If not, the features with larger values will be dominant.
  • If you have too many discrete variables and use dummy coding, distance measures would not work well.

Also, I think my answers for K-means would answer your question of what may happen if we do not normalize features.

Standardizing some features in K-Means


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.