# What exactly does PCA show that I can't figure out otherwise?

First off, apologies if this is basic but I'm still learning about PCA. I am somewhat confused as to what exactly PCA can provide that I can't find out through other means. For example:

1. Dimensionality Reduction/Redundancy: couldn't one just look at the covariance matrix? If we wanted to retain our original feature set but wanted to reduce it in some way, why not just pull out extraneous variables that are highly correlated?
2. Visualization at lower dimensions/clustering: why not just run a clustering algorithm and look at in-cluster and between-cluster statistics?

The only thing I can think of that may be more useful would be if one wanted to use the latent representations or principal components for actual use somehow, like in regression. But even then I'm not sure how common this is, and I never really read about this.

• pca is "just look[ing] at the covariance matrix"; it just allows us to see correlations between more than two of variables at a time :) Nov 5, 2022 at 17:26
• Suppose your data are three columns: one of 1s, one a 50/50 mix of 1s and 0s and one that is 1 minus the second column. Together, these three columns are linearly dependent, but the correlation of each pair is low. // Your PCA-for-regression idea is called partial-least-squares, although I suppose you could naively use PCA first and regression second, for some reason.
– Sycorax
Nov 5, 2022 at 17:35