In cluster analysis, is it better to normalize to $[0, 1]$ (i.e., $\frac{x-\min(x)}{\max(x)-\min(x)}$) the data or standardize via z-score (i.e., $\frac{x-\bar{x}}{s_x}$) it?

I know normalization remove the influence of outliers, while standarization reshape the distribution of the data to a normal one. Are there cases when is better to use one over the other? For example, when doing (1) Hierarchical Clustering, or (2) Particioning clustering.

  • $\begingroup$ Standardization does not reshape the distribution. There are special rank percentile methods of transform which reshape it onto the normal curve. $\endgroup$ – ttnphns Oct 17 '19 at 7:53
  • $\begingroup$ Both standardization and the normalization are used and both have their pros and cons. Just look at the formulas themselves and you will be able to answer which method is better (if needed at all) in your particular case of clustering. $\endgroup$ – ttnphns Oct 17 '19 at 7:59
  • $\begingroup$ Isn't it better to standardize for the sake of computing better euclidean distances? $\endgroup$ – Nip Oct 17 '19 at 15:48

Normalization rarely works better than standardization.

Nevertheless, even standardization is a pretty crude heuristic that should only be used with care.

Either can ruin important properties of your data.

When you need to standardize, often you approach is wrong. You likely have attributes of a very different scale, and the combination of these attributes with some distance does not yield interpretable results. Try to write down what your algorithm really means on your data, then check it this is any use to you. People are always eager to cluster data because they want to have partitions, but neglect that these partitions shouldn't be "random" but should have some interesting properties...

| cite | improve this answer | |
  • $\begingroup$ Does computing euclidean distances count as a need? $\endgroup$ – Nip Oct 17 '19 at 16:02
  • 1
    $\begingroup$ Euclidean distances are a tool, not a need. They may be the wrong tool. They particularly tend to be the wrong tool if you use normalization/standardization because it can completely change their results... $\endgroup$ – Has QUIT--Anony-Mousse Oct 17 '19 at 21:54
  • $\begingroup$ Could you, please, provide some bibliography to learn about when to normalize, standardize, and when not? $\endgroup$ – Nip Oct 18 '19 at 4:28
  • $\begingroup$ I don't know many. If you find any, please share. $\endgroup$ – Has QUIT--Anony-Mousse Oct 18 '19 at 4:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.