Principal Component Analysis and input feature distribution

Question

I am performing PCA on dataset of shape 300,1500 using scikit learn in Python 3. I have following questions in the context of PCA implementation in scikit learn and generally accepted approach.

1) Before doing PCA do I remove highly correlated columns? I have 67 columns which have correlation > 0.9. Does PCA automatically handle this correlation I.e ignores them?

2) Do I need to remove outliers before performing PCA?

2b) if I have to remove outliers how best to approach this. Using z-score for each column when I tried to remove outliers (z-score >3) I am left with only 15 observations. It seems like wrong approach.

3) Finally is there ideal amount of cumulative explained variance which I should be using to choose Principal components. In this case around 150 components give me 90% cum explained variance

Answer 1

1. PCA is a whitening transformation. After applying PCA, all the different principal components are uncorrelated, even if some of the original variables are highly correlated. This question has been discussed here.

2. PCA is sensitive to outliers. This question has been discussed here.

2b. You can always tune your threshold and the number of removed observations will change accordingly. However, your data may not be distributed like a bell-curve, and the distribution may not even be symmetric. You should plot some histogram of your data to get a sense of how your data is distributed, and what your outliers look like.

3. No. This really depends on your data and what you want to do with it after applying PCA. This has been discussed here.