-1
$\begingroup$

How would the eigen values look like when we apply PCA to a dataset with zero correlation between variables and when there is very high correlation between variables

$\endgroup$

migrated from mathoverflow.net Dec 24 '18 at 15:59

This question came from our site for professional mathematicians.

  • 3
    $\begingroup$ this looks like a homework question. what have you learned from your initial research? $\endgroup$ – honi Dec 24 '18 at 16:02
0
$\begingroup$

If some data set has literally zero correlation between its variables then its correlation matrix is literally the identity matrix, which has eigenvalues of all 1's for as many columns as there are in the matrix. If all the eigenvalues are the same then doing PCA will not help you much in terms of dimension reduction, because the first principal component will not account for any more variation then the following principal components, because eigenvalues literally mean how much variation in the data is accounted for by the corresponding principal component.

As the amount of correlation increases, then the first eigenvalue will slowly start to increase, and the later ones will decrease to 'compensate.' As such, PCA will be better performed on highly correlated data.

Here is some R code if you'd like to see it for yourself.

    no.cor<-rbind(c(1,0,0),c(0,1,0),c(0,0,1))
    eigen(no.cor)
    low.cor<-rbind(c(1,0.1,0.1),c(0.1,1,0.1),c(0.1,0.1,1))
    eigen(low.cor)
    high.cor<-rbind(c(1,0.8,0.8),c(0.8,1,0.8),c(0.8,0.8,1))
    eigen(high.cor)
$\endgroup$
  • $\begingroup$ I believe the second question is a little subtler than your answer might suggest, because more than one eigenvalue can increase. What happens depends on precisely what "very high correlation between variables" means. Perhaps, then, you could include in your answer some indication of your interpretation of this phrase. $\endgroup$ – whuber Dec 24 '18 at 19:40

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.