0
$\begingroup$

I am running PCA on my data and my KMO value > 0.80 and p-value of Bartlett's test of sphericity < 0.05.

However, the determinant of the correlation matrix ( around 10^-30) is very close to zero.

I tried to remove some columns with high correlation coefficients, but the determinant still remained relatively very close to zero.

Will the presence of linearly dependant variables hinder the reliability of PCA?

Improve this question
$\endgroup$
6
  • $\begingroup$ What are the dimensions of your dataset? $\endgroup$
    – user289381
    Jul 3, 2020 at 21:25
  • $\begingroup$ I have a 50 by 50 dataset. $\endgroup$
    – user05
    Jul 4, 2020 at 14:44
  • $\begingroup$ You can try to regularize the covariance matrix with Ledoit Wolf shrinkage. You can find here an example in Python scikit-learn.org/stable/auto_examples/covariance/… $\endgroup$
    – user289381
    Jul 4, 2020 at 14:50
  • $\begingroup$ In general, having correlated variables is not a problem for PCA. You will have some loadings with weights coming from these variables only. PCA is used to compress the redundant information (from covarying variables) in a single dimension. $\endgroup$
    – user289381
    Jul 4, 2020 at 14:56
  • 1
    $\begingroup$ The whole point of PCA is to analyze correlation matrices with zero or tiny determinant. If the determinant were noticeably nonzero, you could immediately conclude that the data are nearly spherical and that PCA would accomplish little or nothing. See the many examples at stats.stackexchange.com/questions/2691/… and ponder what the determinants might be for each one. $\endgroup$
    – whuber ♦
    Jul 4, 2020 at 15:17

1 Answer 1

Reset to default
1
$\begingroup$

No, it is not a problem. Even if the determinant where exactly zero, pca can/will be useful. The point of PCA is to find linear combinations in the data with high variance, and the usefulness (or existence) of such is not contradicted by other linear combinations with low variance.

See the comment by user whuber:

The whole point of PCA is to analyze correlation matrices with zero or tiny determinant. If the determinant were noticeably nonzero, you could immediately conclude that the data are nearly spherical and that PCA would accomplish little or nothing. See the many examples at Making sense of principal component analysis, eigenvectors & eigenvalues and ponder what the determinants might be for each one.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.