# How to compare clustering results between "raw" and normalized data

I have a dataset and I would like to apply a clustering algorithm to find some groups. I do not have any label, so it is just wondering if I can find relevant clusters. If it may help, it is information about some players (say age, height, weight, and some other performance metrics - the point is there are features with different scalings...). Also, although I think my question applies to any clustering algorithm, I am currently using K-Means.

Now, I just applied a K-Means with 5 clusters on my data and I calculated some very standard performance metrics: inertia and silhouette score (overall average, and average per cluster).

Then, given that scaling data is considered a best practice, I decided to standardize every feature (z-score) and apply again K-Means with 5 clusters. I computed the same metrics. I expected inertia to be lower (it is a sum of squared distances...) but surprisingly all the silhouette scores were lower, too! I expected silhouette to improve on scaled data. ¿Why wasn't that the case?

This led me to ask some questions: ¿is it enough to conclude that in this case it is better not to scale the data or am I missing something? Can we actually compare these two silhouette scores, despite being computed on data with different scalings? If not, which would be a better approach to evaluation / comparison?

Cheers!

• See the table in the end of this answer: stats.stackexchange.com/a/358937/3277. Its column "To different linear transform of scale variables". Z-score of a feature is such linear transform, different for each feature. Check which cluster validity index is insensitive to it. Use such ones to compare your results. Jun 13, 2021 at 10:22
• So, it turns out that of the listed there indices, you can use Log det ratio, Ratkowski-Lance, and AIC, BIC. You cannot use Silhouette, any of its versions. Jun 13, 2021 at 10:29
• If by inertia you mean the SSwithin "elbow" index, you can't use it either, for your task to directly compare quality of clustering results based on scaled vs raw features. Jun 13, 2021 at 10:34

There are different considerations here. The Euclidean distances between points are different with and without scaling. The "optimal" choice of distances depends on the meaning of the data and what should qualify data to be in the same cluster. Scaling will basically unify the influence of the different variables on the clustering, which is very often a reasonable thing to do, particularly if the variables have different scales, the values of which cannot directly be compared. If I understand your data correctly, scaling looks like a sensible thing to do, although in general it also depends on the aim and potential use of the resulting clusters. Note also that there are various ways of standardising the data, instead of z-scores one can also use median and MAD scaling or scaling to 0-1 range, which in particular treat outliers differently (there is no clear right and wrong here unfortunately, with or without outliers all these have pros and cons).

Regarding inertia (I guess you mean what I'd call within-cluster mean squares), inertia values cannot be compared between different distances, so this doesn't tell you anything about which clustering is better.

Regarding the silhouette (I'll use ASW for the average silhouette width), the situation is more complicated. These are scaled so that in principle they could be compared between clusterings based on different distances, however note that quality of the clustering (as measured by ASW) is different from the appropriateness of the distance for the application.

It may be that in reality in fact the data are not meaningfully clustered but pretty homogeneous, or rather that their variation doesn't manifest itself in a grouping. In this case a worse clustering in the sense of ASW on the more appropriate distance can be more appropriate (in the sense of giving you better information about the distribution of players) than a strong clustering (ASW) on a less appropriate distance.

The thing to understand here is that the distance should not be chosen in order to optimise the clustering quality.

I give two examples to illustrate that:

(1) If you really want a distance that optimises clustering quality, and, say, you want 5 clusters, the best thing to do would be to assign the distance so that there are five clusters with distance of 1 between each two points in different clusters and zero within each cluster. This would produce a maximum ASW of 1, and chances are you can find a weird nonlinear transformation of the original variables that delivers you such a thing or something close to it, however nobody would think that this is a good way to arrive at a meaningful clustering of the data. Therefore optimisation of the ASW by choice of distance is useless.

(2) Let's say you have 20 variables, and the first one, V1, is measured so that it has a variance of 1000, whereas the 19 others have a variance of 1. Let's also say that if you look at the variables separately, V1 by accident is the one that is most clearly clustered, however there is also (if weaker) clustering structure in the other variables, that at least in part differs from the clustering in V1. If you don't standardise, V1 will dominate the clustering totally and the other 19 variables will have an impact pretty close to zero on the clustering, achieving a good ASW. If you standardise, what you implicitly do is to weight V1 down and the others up, so that the potential impact of the variables on the clustering is unified. In fact, V1, having the strongest clustering structure, will still probably have the most impact, but that may mean 10% or so, rather than almost 100% without scaling. The ASW will probably be worse because the overall clustering structure is not as pronounced as on V1 alone, but surely if you collect 20 variables and all of them are meaningful and important from a subject-matter point of view, you don't want a clustering that is essentially based on one variable alone! You should rather accept the somewhat less sharp grouping that you get taking all your variables into account, which just reflects the overall information more appropriately than a clustering based on one variable only, even though the latter has the better ASW.

(Note that nothing of this discussion stops you from using the ASW to compare clusterings based on the same distance.)

• Thanks! Just to be sure I got it right, "distance" is what I meant with scaling, i.e. the way the data is scaled, right? I guess it was important for me to understand that the distance should not be chosen in order to optimise the clustering quality. To summarize I'd say that unless we have a specific reason to weight some features more it is more sound to scale data so to homogenize their impact even at the expense of ASW. However there is no a definitive way to compare the results on the two "distances", nor to measure the tradeoff between "sharpness" vs "soundness" (ASW vs ???). Jun 7, 2021 at 22:40
• The distance is the Euclidean distance used within the k-means objective function (which has a squared distance between every observation and the cluster mean in it, and can be rewritten in terms of pairwise distances between the observations). This distance of course is different with or without scaling. Jun 7, 2021 at 22:46
• The question whether to scale is difficult as there is a reason not to do it if the intended information content of a variable is proportional to its variance. If data are measured on clearly different and incompatible scales, however, scaling is almost always better than not scaling (although in some situations transformations such as taking logs can be even better, at least as first step). Jun 7, 2021 at 22:50
• "Soundness" is unfortunately a very context dependent concept and therefore not measurable in a general formal way. In some situations one can measure the connection of a resulting clustering (or the distance) to an external variable, or have expert assessments to validate either the distance or the clustering against. I would probably not use the ASW to compare across different distances, despite it being defined on some kind of "absolute" scale. Jun 7, 2021 at 22:53
• I have a data from 2nd example, where the first variable is bigger than the other. But I don't understand, how I should calculate mean values for data standartisation: averaging all variables of one objects (to get one average for every object), or averaging all values of every veriable (to get one average for every variable)? Jun 13 at 7:48