Under what conditions would clustering on top of Principal Components would return different result (and worse) than clustering on the data itself?

Question

Since Principal components capture most of the information, clustering on them should provide similar result as that of the clustering on the original data.

As such, it seems to me (who's not a statistician, but interested nonetheless) like principal components would be better suited to showcase natively existing clusters since the collinearity would be eliminated.

But are there situations where clustering on PCs may not be as good and may provide worse results than the original data set?

I can think of a situation where, having many correlated columns and the cluster having to be biased towards this component, can yield a worse result. Is this a common occurrence? If so, this seems to me like distributing weights without actually understanding it.

Can an expert throw some light to intuitively understand?

Answer 1

It is common to scale data when doing PCA.

Usually, you use the eigenvalues to scale each component by $1/\sqrt{\lambda_i}$.

Any rescaling of the data does have a massive effect on the results.

PCA can serve as a heuristic if you have many (preferrably continuous - the use of PCA on binary attributes is somewhat questionable) attributes of different scale, and you

1. expect strong correlations to be present in the data
2. do not have information on how to properly rescale and weight the individual attributes

In the rare scenario where all attributes have the same importance and scale, PCA becomes much more well-founded. As a rule of thumb: if a variance of $x$ in every attribute has the exact same importance (say because the attributes are coordinates in your measurement chamber) then the use of PCA is strongly backed by theory. If attribute 1 is the shoe size, and attribute 2 is income, then PCA will be ruined by the differences in scale, and the outliers in income.