0
$\begingroup$

I have some questions about KMeans that I would like to discuss.

I have several features, the minimum and maximum values ​​between columns vary, so I applied the "MinMaxScaler" transformation to the data before applying the KMeans algorithm. However, the result (Silhouette Score) is better if I don't consider ANY transformation. Later, I tested with other transformations ("StandardScaler", first "MinMaxScaler" and then "StandardScaler", and "Normalizer") and "Normalizer" gives the best result (Silhouette Score).

These are my questions:

  • Having different maximum and minimum intervals between features, would you choose NOT to apply ANY transformation if your Silhouette Score gives a higher value than applying any other transformation? Or would you necessarily apply some transformation?

  • The other question is: is it reasonable to apply "Normalizer" in KMeans? and if so, how could you apply the inverse transformation? I get the following error when applying the inverse transformation: "AttributeError: 'Normalizer' object has no attribute 'inverse_transform'"

Thanks a lot.

$\endgroup$
4
  • $\begingroup$ Your terminology seems to refer to a specific software. Although I can guess what "MinMixSacler", "Normalizer" etc. mean, it is not clear what they are unless you explain them. $\endgroup$ Commented Aug 3, 2021 at 21:39
  • $\begingroup$ They are transformations (benalexkeen.com/feature-scaling-with-scikit-learn) that must be applied to features, I understand that they should be used mainly in distance-based algorithms. But, on some site recommend testing with several transformations, including WITHOUT transformation, and opting for the one with the best result, I don't know if that is correct. $\endgroup$
    – user140259
    Commented Aug 3, 2021 at 22:02
  • 1
    $\begingroup$ OK, I updated my answer with this info. Choosing the "best" result according to the Silhouette Index is not a good idea in my view, see my answer. (Not many understand this, so unfortunately you find it recommended in some places where people think that the data should decide everything on their own and meaning does not matter. But usually it does!) $\endgroup$ Commented Aug 3, 2021 at 22:13
  • $\begingroup$ You should not pick a transform basing the choice on a cluster validity index maximization (Silhouette or other). Transformation should be done consciously, with reflection of what you are doing and why. Any transform is its own theoretical load. $\endgroup$
    – ttnphns
    Commented Aug 3, 2021 at 22:39

1 Answer 1

5
$\begingroup$

Generally scaling serves to give variables in some well defined sense (which differs between different methods) the same influence on the clustering. Whether you should scale or not depends on the meaning of the variables and the aim of clustering. If your variables have incompatible measurement units, scaling is a good idea. However, if for some reason (which can occur in practice) different ranges of variables imply that the variables with larger ranges are more informative, you should not scale, as scaling removes this information from the analysis. An example are 5-point Likert scales from different questions. If one question has answers between 1-5 and another only has 3 and 4, it may be that people are indifferent on the second question, and the first question will rightly have more influence on the clustering if you don't scale. (This may be different if for some reason you think the second question is of key importance, even the difference between 3 and 4.)

There are various different methods of scaling. MinMaxScaler apparently is linearly scaling to a 0-1 range; another standard method is to standardise to zero mean and unit variance (StandardScaler). Both of these (and others) have their pros and cons. Scaling to 0-1 range may basically destroy the information in a variable that has a gross outlier; however zero mean/unit variance scaling may give an outlier more influence on the clustering after scaling. In some literature, 0-1 range scaling was called "more appropriate" for clustering but I think it depends on the data. Sec. 3.4 of this has some more details:

C. Hennig: Clustering Strategy and Method Selection (from the "Handbook of Cluster Analysis, link to free version on arxiv)

"Normalize" is an entirely different approach, not scaling the variables for having equal influence but rather scaling over observations. Again, it entirely depends on the meaning of the data if that's appropriate; in my experience it rarely is.

I generally do not recommend to choose the scaling by optimising the Silhouette Score or similar. The reason is that the variables as they are used for clustering define the meaning of the clusters. If you wanted to transform the data in such a way that they give you an "optimal" clustering with $K$ clusters according to the Silhouette Score, you should just transform them to $K$ different points, and then you'd have no within-cluster variance and perfect clusters that don't mean anything. If you want meaningful clusters, you need to scale your variables in such a way that this respects the influence that they are supposed to have on the clustering for it to carry the meaning that you want it to have. If the clusters that you then get are not that clear (not a high Silhouette Score) this is just how it is, and you can communicate that. Better to reflect the meaning in your data rather than having a formally nice solution that has ignored the information about the meaning of the variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.