1
$\begingroup$

I calculated PCs for my samples and I am showing here data frame that has samples as my rows and PCs as my columns. My question is in order to decide on the number of PCs to keep for my regression analysis is this valid approach?

> head(a)
       PC1      PC2        PC3       PC4      PC5        PC6       PC7
1 -13.0692 3.825460 -2.8089500 -0.120865 -9.53690  2.2582600  0.975514
2 -13.0419 4.076040 -2.3597900  2.326170 -0.73101 -1.5689400  1.642810
3  -9.5570 4.270540 -0.9153700 -0.160893 -2.27807 -1.0854500 -0.551797
4 -11.4407 0.716765 -0.0932982 -1.229210  2.56851 -0.0708945  2.841000
5 -15.0062 6.971110 -2.9324700 -3.033660 -3.73211  1.8029200  0.712720
6 -13.8156 1.667130 -1.2647800  3.929120  4.12255  0.2541560  1.119040
    PC8      PC9      PC10
1 -2.220460  1.15324  3.677270
2 -2.552010 -2.57720  0.111892
3  0.360637  0.30142 -1.288880
4  1.391550 -5.13552 -1.975630
5  1.937330 -1.83419 -1.462170
6 -0.637011 -3.15796 -1.238350
...

a.cov <- cov(a)
a.eigen <- eigen(a.cov)
PVE <- a.eigen$values / sum(a.eigen$values)

> PVE
  [1] 0.49967626 0.22981763 0.07138644 0.04307668 0.03680999 0.02830493
  [7] 0.02526709 0.02384502 0.02135397 0.02046199

So it seems that the first 4 PCs explain about 85% of my variance. Is this the valid way on how to go abotu deciding the number of PCs to keep?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ It is if the primary requirement of your model is to explain an amount of variance less than or equal to 85%. There are many different stopping criteria depending on your intentions for the model. $\endgroup$ – ReneBt Jul 25 at 5:59
0
$\begingroup$

Yes, typically this is a good way to select how many principal components to include in your model.

It could help to visualize the eigenvalues as well. Plot them from highest to lowest and find the point where the curve flattens out (so that later eigenvalues make less impact on the information content)

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks! I guess I can do "scree" plot in R for the purpose of visualization of PCs $\endgroup$ – anamaria Jul 25 at 4:05
  • $\begingroup$ @anamaria glad I could help! Another thing to consider is what the data represent. For example in “factor analysis”, there is a theoretical model underlying the data which suggests that a certain number of components comprise the data. See, for example the “big 5” personality models in which 5 components are selected, regardless of pct. variance they explain. But for most purposes, percent of variance explained is the benchmark for model selection with PCA. $\endgroup$ – phil Jul 25 at 4:18
  • $\begingroup$ Yes I was looking into using fa.diagram() from psych package as described here: dwoll.de/rexrepos/posts/multFA.html But I 'm having trouble explaining what factors on diagram mean, and why some are in "red". Do you have by any chance any link that helps interpret factor diagram? $\endgroup$ – anamaria Jul 25 at 4:39
  • $\begingroup$ I don’t know any other sources for the factor diagram unfortunately. I think you are referring to the fa.diagram() from that link, which gives a bipartite graph of the variables and their weights (“loadings”) in each of the factors/components. In that case, the diagram shows which variables weigh into each component, and how much. In that example, the threshold is 0.5, so there is a line between variable v and component MR1 only if the entry in MR1 corresponding to v is greater than 0.5 in absolute value. The red lines are associated with negative weights in the component. $\endgroup$ – phil Jul 25 at 4:57
  • $\begingroup$ Just a note, some of your components (eg the first one) are all negative. This could give a diagram with a lot of red lines. However, you can multiply the component by -1 to make all entries positive, and it will still be valid due to the properties of eigenvectors $\endgroup$ – phil Jul 25 at 5:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.