I performed factor analysis on R using factanal. Following advice I found on this tutorial, I chose the number of factors as being the number of principal components that capture 90% of the variability. I got the following table of results:

               Factor1 Factor2 Factor3 Factor4 Factor5 Factor6 Factor7 Factor8 Factor9
SS loadings     29.677   9.530   6.340   5.638   4.188   3.636   1.484   0.804   0.651
Proportion Var   0.309   0.099   0.066   0.059   0.044   0.038   0.015   0.008   0.007
Cumulative Var   0.309   0.408   0.474   0.533   0.577   0.615   0.630   0.638   0.645

Test of the hypothesis that 9 factors are sufficient.
The chi square statistic is 30148.02 on 3732 degrees of freedom.
The p-value is 0 

I tried with one factor and I got a p-value of 0 as well...

  1. Why is the cumulative var smaller than with 9 PCs for a pca?
  2. How can I interpret the above table to choose the correct number of factors? (since the p-value is the same for 1 of 9 factors)

There are many similarities between PCA and Factor Analysis, and often insights from one can inform the other. Here though, the difference in their objectives is leading to the discrepancy you observe. Quoting this comparison of PCA and FA

PCA results in principal components that account for a maximal amount of variance for observed variables; FA account for common variance in the data

The point I want to make is that PCA maximizes the amount of variance explained in its loadings, but FA does not exactly do that. This paper makes a case for why, with this in mind, it is more realistic to select the number of factors for FA based not on the percentage of explained variance, but rather the explained common variance, which is more in line with the spirit of FA.

Speaking to your second question: If you truly want to explain 90% of the variation in your data, try to run the analysis again with more factors and see how many you need to get the cumulative variation up to 90%. This however, may not work for the reason given by @ttnphns. Maybe you can also make a scree plot and see if you find an elbow. That could also give you a good idea of the number of components to take. Based on the table you show, it may be that adding more factors will not help much (since the change between the variation explained by the last few factors is small). In the end though, perhaps explained common variance will be more appropriate for the FA you are doing.

| cite | improve this answer | |
  • $\begingroup$ Thus PCA will tend to need fewer PC's than FA needs factors to explain the same amount of variation in the data This statement is a bit slippery. Since FA explains only common variability it thus has its ceiling limiting variation below 100% that it can explain, while for PCA the limit is 100%. $\endgroup$ – ttnphns May 28 '14 at 12:55
  • $\begingroup$ @ttnphns : thanks, good point. I looked a bit more and found a source for an alternative to explained variance that is more appropriate for FA. $\endgroup$ – Deathkill14 May 28 '14 at 13:08
  • $\begingroup$ What about the value of the chi square statistic? Isn't it very large? $\endgroup$ – bigTree May 28 '14 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.