I'm doing PCA to analyze a dataset. The dataset size is 10^7 rows, and I have about 2,000 features. My analysis shows that no single principal component captures more than 1% of the dataset's variance. To capture 75% of the variance, I have to use 1,000 principal components! My interpretation of this is that the data might be just noise with very little signal. Is this the correct way of thinking about it?

Here is the plot of the cumulative variance

enter image description here

and a graph of the raw variance values: enter image description here

  • 3
    $\begingroup$ "I need 1000 PCs to capture 75% of the variance." Could you elaborate on why you interpret 0.75 as 75% but 1.00 as 1%? $\endgroup$ – chainD Oct 29 '17 at 20:05
  • 1
    $\begingroup$ I don't have time to answer this properly but just to move you away from the "my dataset is just noise"-idea: Remember that ultimately PCA finds a rotation (read linear combination) of your original data such that their covariance matrix is diagonal (read the projected data are orthogonal). If the original data are already "somewhat independent"/orthogonal with each other, PCA won't help a lot to reduce the number of dimensions (read features) you need to encapsulate the variance of your dataset. $\endgroup$ – usεr11852 Oct 29 '17 at 21:34
  • 2
    $\begingroup$ Your scree-plot is not very smooth, there is good reason to suppose tentatively that 4 or 5 latent factors drive covariations in the data. Jointly these factors are really weak, probably explaining about somewhwere 10% of total variance. But they are good "signal". The rest of the total variance is then variable-specific; whether it is noise or valuable variability depend on what is "noise" for you. $\endgroup$ – ttnphns Oct 30 '17 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.