Clustering and why it might not be a good idea to normalize your data?

Question

I came across this post on Quora and the first answer cleared up why it might be a good idea to always normalize your data, but I want to understand what happens conversely. More specifically, what types of negative impacts would normalization of data have when using clustering?

Answer 1

In some applications the measurement units of different variables are the same, and the meaning is actually pretty much the same. Normalisation is bad if in such a situation the variance is actually informative in the sense that a larger variance implies that the variable is more relevant. One such example that I have come across is voltage traces on different electrodes left by combustion of particles. An electrode with a low variance just isn't informative, it is dominated by noise, and the most relevant voltage signal on an electrode will correspond to the largest variance. If such data are normalised, the uninformative variables with the low variances will have their variances increased, so the noise will be amplified, whereas the more relevant variables will be relatively weighted down, which is undesirable and will make the clustering more dominated by noise. In this situation the different variables all correspond to the same kinds of measurements on the same scale, which suggests that normalisation is not necessarily required, however this doesn't suffice for it necessarily being bad. What makes it bad is the knowledge (or at least suspicion; these things are often not known for sure but rather suspected) that the clustering-relevant information is connected to a higher variance.

Another conceivable example are situations where some people are asked to rate different items (films, say) on a sufficiently detailed scale. If the aim is to cluster the users, the importance of the films to differentiate between them should be expected to be strongly connected to the variance; on a film with low variance opinions are just not that divided, and it would be a bad idea to make ratings on such films relatively more influential for clustering by normalisation.

More generally, abstract examples can be easily constructed in which large variance is caused by large distances between clusters, making such variables more informative for clustering. Consequently, normalising will be counterproductive in such cases as it will artificially decrease the between-cluster distances on these variables. This however is hard to diagnose unless the dimensionality is very low and things can be clearly seen on scatterplots, as it in principle requires the clusters to be already known, which is not normally the case.

But this illustrates that normalising is not always good; it is a simple (often fine but not always the best) remedy in cases in which the variables have wildly different variances and/or incompatible measurement scales, without any reason to believe that a larger variance may point to more information that a variable holds for clustering.