How would the eigen values look like when we apply PCA to a dataset with zero correlation between variables and when there is very high correlation between variables
1
-
3$\begingroup$ this looks like a homework question. what have you learned from your initial research? $\endgroup$– honiDec 24, 2018 at 16:02
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
If some data set has literally zero correlation between its variables then its correlation matrix is literally the identity matrix, which has eigenvalues of all 1's for as many columns as there are in the matrix. If all the eigenvalues are the same then doing PCA will not help you much in terms of dimension reduction, because the first principal component will not account for any more variation then the following principal components, because eigenvalues literally mean how much variation in the data is accounted for by the corresponding principal component.
As the amount of correlation increases, then the first eigenvalue will slowly start to increase, and the later ones will decrease to 'compensate.' As such, PCA will be better performed on highly correlated data.
Here is some R code if you'd like to see it for yourself.
no.cor<-rbind(c(1,0,0),c(0,1,0),c(0,0,1))
eigen(no.cor)
low.cor<-rbind(c(1,0.1,0.1),c(0.1,1,0.1),c(0.1,0.1,1))
eigen(low.cor)
high.cor<-rbind(c(1,0.8,0.8),c(0.8,1,0.8),c(0.8,0.8,1))
eigen(high.cor)
-
$\begingroup$ I believe the second question is a little subtler than your answer might suggest, because more than one eigenvalue can increase. What happens depends on precisely what "very high correlation between variables" means. Perhaps, then, you could include in your answer some indication of your interpretation of this phrase. $\endgroup$– whuber ♦Dec 24, 2018 at 19:40