I have a financial dataset with 9500 observations with 4000 variables. There are around 2500 variables highly correlated (higher than 0.95). Without removing any variable, I have applied PCA. According to my results, first 100 components explain 78.8%. When I check the PCs in detail, I observed that the highest loadings range between 0.01 - 0.05. On the other hand, the original variables having the highest loadings make sense. That is, same group of variables are appearing on the top (top positive) or on the bottom for PCs (bottom negative).

Besides the approach above, I have removed one variable from the pairs with a correlation higher than 0.95.This reduced my variable set to 1200. Then I again applied PCA. However, there happened no significant change on loadings.

In short, how should I pick the most important components for each PC when such very low loadings present?

• Is it possible you have some co-linear variables? Perhaps checking for this among your variables that are highly correlated would be a good sanity check.
– guy
Dec 11 '17 at 13:48
• Yes @guy. I have many variables even having a correlation of 1. I have removed one variable from those pairs having very high correlations and applied pca again. However, I did not observe a significant increase in coefficients. Dec 11 '17 at 19:29
• Are you speaking of loadings or of eigenvectors? stats.stackexchange.com/q/143905/3277. Also, did you do the analysis based on correlations or on covariances? If the latter, how big are variances in your matrix? Dec 12 '17 at 1:50

RotatedCoefficients = rotatefactors(PCACoefficients(:,5));