What are the pros and cons of using K-means centroids (using cluster centroids as samples) vs Principal Component Analysis (PCA) as a preprocessing stage in machine learning? In what situation is their use recommended?
Unless medoids are used, both will destroy the features because the original samples will be transformed.
Whereas PCA reduces dimensionality, the method you named cluster centriods in your link performs subsampling.
PCA removes features, cluster centriods remove datapoints.
PCA is thus used in case you have a lot of features, cluster centroids for subsampling is used in case you have a lot of samples (another advantages could be to reduce imbalance as a large cluster and a small cluster both results in a single cluster centriod).
Basically, they are completely different approaches to dimensionality reduction and in practice I can't say I have seen a golden rule to use either of them. But here is explanation for both:
You can think of PCA as finding a linear combination of features such that variance of data is maximized in that combination. So if you suspect that there are linear combination(s) that describe your data -- use PCA.
KMeans on the other hand tries to cluster your data together. Which means that it tries to find centroids such that mean distance is minimal. When we use KMeans as transform we just take distance to centroids. So, if you suspect that data clumps into clusters -- use KMeans.
You can even use both of them together. E.g. Use PCA and then KMeans.
Try which one works better and stick to that.
