I am running PCA on my data and my KMO value > 0.80 and p-value of Bartlett's test of sphericity < 0.05.

However, the determinant of the correlation matrix ( around 10^-30) is very close to zero.

I tried to remove some columns with high correlation coefficients, but the determinant still remained relatively very close to zero.

Will the presence of linearly dependant variables hinder the reliability of PCA?

  • $\begingroup$ What are the dimensions of your dataset? $\endgroup$ – user289381 Jul 3 at 21:25
  • $\begingroup$ I have a 50 by 50 dataset. $\endgroup$ – user05 Jul 4 at 14:44
  • $\begingroup$ You can try to regularize the covariance matrix with Ledoit Wolf shrinkage. You can find here an example in Python scikit-learn.org/stable/auto_examples/covariance/… $\endgroup$ – user289381 Jul 4 at 14:50
  • $\begingroup$ In general, having correlated variables is not a problem for PCA. You will have some loadings with weights coming from these variables only. PCA is used to compress the redundant information (from covarying variables) in a single dimension. $\endgroup$ – user289381 Jul 4 at 14:56
  • 1
    $\begingroup$ The whole point of PCA is to analyze correlation matrices with zero or tiny determinant. If the determinant were noticeably nonzero, you could immediately conclude that the data are nearly spherical and that PCA would accomplish little or nothing. See the many examples at stats.stackexchange.com/questions/2691/… and ponder what the determinants might be for each one. $\endgroup$ – whuber Jul 4 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.