I've heard statements like this many times over the years, and it's perhaps expressed most clearly by Preacher & MacCullum (2003), which is a popular paper on stats.stackexchange.com (e.g. mentioned twice in this question thread). Preacher & MacCullum write on p20 that

PCA does not explicitly model error variance, which renders substantive interpretation of components problematic. This is a problem that was recognized over 60 years ago (Cureton, 1939; Thurstone, 1935; Wilson & Worcester, 1939; Wolfle, 1940), but misunderstandings of the significance of this basic difference between PCA and EFA still persist in the literature.

I could not find all these old papers, but the Wilson and Worcester (1939) one did now allow me to reach a clear conclusion about why failing to explicitly model error variance should make the substantive interpretation of components problematic.

  • Cureton, E. E. (1939). The principal compulsions of factor analysts. Harvard Educational Review, 9, 287-295.

  • Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift's electric factor analysis machine. Understanding statistics: Statistical issues in psychology, education, and the social sciences, 2(1), 13-43.

  • Thurstone, L. L. (1940). Current issues in factor analysis. Psychological Bulletin, 37(4), 189.

  • Wilson, E. B., & Worcester, J. (1939). Note on factor analysis. Psychometrika, 4(2), 133-148. Chicago.

  • Wolfle, D. (1940). Factor analysis to 1940. Psychometric Monographs.

  • $\begingroup$ "EFA vs PCA" is a very extensively discussed topic; and on this site, too. Read threads stats.stackexchange.com/q/1576/3277, stats.stackexchange.com/q/123063/3277, stats.stackexchange.com/q/94048/3277, and other (inspect links in the comments there). Issue of "interpretation" is touched e.g. in stats.stackexchange.com/a/123089/3277. $\endgroup$
    – ttnphns
    Mar 15 '17 at 8:48
  • 1
    $\begingroup$ @amoeba I edited the OP to give an example. $\endgroup$ Mar 15 '17 at 9:06
  • 1
    $\begingroup$ PCA is different from FA, but IMHO this particular distinction is a red herring. PCA "does not explicitly model error variance" means that PCA is not a probabilistic model. That's true in its standard formulation, but PPCA is a probabilistic model, and is mathematically equivalent to PCA. I wrote a lot about that in my answer here stats.stackexchange.com/questions/123063. Looking now in the Preacher & McCallum, I see a footnote on the same page claiming that in FA "models are testable" whereas in PCA "they are not". Again, wrong/misleading: one can test PPCA as well as FA. $\endgroup$
    – amoeba
    Mar 15 '17 at 9:21
  • $\begingroup$ user1205901, FA not only models error variance, it actually models the correlation (or covariance matrix), more directly or less directly - depending on the extraction method. $\endgroup$
    – ttnphns
    Mar 15 '17 at 9:32
  • $\begingroup$ @amoeba, Btw, can you suggest me (a lazy one) a ready-made pseudocode(s) or understandible code or clearly described algorithm of PPCA? I'd wish maybe to rewrite it into SPSS syntax, w/o "inventing" it from scratch. Can you? Thanks. $\endgroup$
    – ttnphns
    Mar 15 '17 at 9:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.