# Using PCA to perform feature selection when variability and correlation are the only selection critera?

I'm running some data exploration and have a large ish number of variables with varying degrees of correlation and covariance. I'd like to start throwing out some variables I don't "need" according to the criteria that most variable and uncorrelated ones are my most important *** (I know this isn't true in general!). I know pca is a poor feature selection choice, and I really only use it for visualization. However, given my criteria, is this a proper application?

It's been awhile since some of my instructional days and I recall a reasonable amount of the linear algebra behind it (using orthogonal decomposition and using the eigenspectrum of a matrix to create a hyper plane whose axes are the directions of maximum variability, the linear scoring used to create the pcs etc etc).

My big question though- Is it necessarily true that high loading scores on a set of factors imply that these factors account for the most covariance in a technical sense (i.e. these are the factors where things are 'most different'). If the former is true- and I want to compare loading across multiple pcs in this sense, where is my breakpoint (i.e suppose I have a low loading on pc 1 for factor A and a high loading for factor B on pc2- if the explained variance due to pc1 is say 70 percent, how can I deduce that more of my data points vary wrt to B than A).

Thanks for all your input. It's been a long day so please let me know where I can clean this up if needed.

• Sorry I apparently come from a different subfield -- what are "loadings?" And to clarify, you want to keep the variables (not the dimensions in data space) that have the least correlation with (something) Commented Oct 5, 2017 at 13:15
• no worries! people sometimes use scores and loadings interchangeably which makes things...confusing to say the least. loadings in this context is each variables contribution to the the principle component as in the context of their linear combination in creating the PC Commented Oct 5, 2017 at 16:49