Is the Kaiser rule (choosing only PCs with eigenvalues >1, see e.g. here on Wikipedia) a reasonable method to select the number of principal components to retain? What are its pros and cons?
$\begingroup$
$\endgroup$
8
-
8$\begingroup$ Are you asking the same question over and over again, every time erasing the previous version? $\endgroup$– amoebaDec 29, 2016 at 15:07
-
2$\begingroup$ Best for what purpose? $\endgroup$– Matthew DruryDec 29, 2016 at 18:23
-
4$\begingroup$ Yes, but "best principal components" is not an inherent property of the universe we live in. "best" must be defined relative to the problem you are attempting to solve. As asked this is unanswerable, you need to add much more context. $\endgroup$– Matthew DruryDec 29, 2016 at 18:28
-
4$\begingroup$ Please stop deleting & reposting the same question. If there are any issues, resolve them on this thread. $\endgroup$– gung - Reinstate MonicaDec 29, 2016 at 19:15
-
2$\begingroup$ I would not close this as unclear or too broad. "Best" sounds vague, but people routinely select some number of PCs as "interesting" and Kaiser's rule is one way to select this number. Asking about possible pitfalls of this method sounds a like a fine question to me (+1). So I'd say with some editing this Q could stay. @MatthewDrury $\endgroup$– amoebaDec 29, 2016 at 22:52
|
Show 3 more comments
1 Answer
$\begingroup$
$\endgroup$
1
The advantage of the rule is that it is easy to calculate, especially if you live in the 1950s, and don't have access to a fast computer.
The disadvantages ... well, I'm going to quote Preacher and MacCallum, in their paper "Repairing Tom Swift’s Electric Factor Analysis Machine". It's worth reading the whole paper, available here: http://www.quantpsy.org/pubs/preacher_maccallum_2003.pdf
"... use of the rule in practice is problematic for several reasons. First, Guttman’s proof regarding the weakest lower bound applies to the population correlation matrix and assumes that the model holds exactly in the population with m factors. In practice, of course, the population correlation matrix is not available and the model will not hold exactly. Application of the rule to a sample correlation matrix under conditions of imperfect model fit represents circumstances under which the theoretical foundation of the rule is no longer applicable. Second, the Kaiser criterion is appropriately applied to eigenvalues of the unreduced correlation matrix rather than to those of the reduced correlation matrix. In practice, the criterion is often misapplied to eigenvalues of a reduced correlation matrix. Third, Gorsuch (1983) noted that many researchers interpret the Kaiser criterion as the actual number of factors to retain rather than as a lower bound for the number of factors. In addition, other researchers have found that the criterion underestimates (Cattell & Vogelmann, 1977; Cliff, 1988; Humphreys, 1964) or overestimates (Browne, 1968; Cattell & Vogelmann, 1977; Horn, 1965; Lee & Comrey, 1979; Linn, 1968; Revelle & Rocklin, 1979; Yeomans & Golder, 1982;Zwick & Velicer, 1982) the number of factors that should be retained. It has also been demonstrated that the number of factors suggested by the Kaiser criterion is dependent on the number of variables (Gorsuch, 1983; Yeomans & Golder, 1982; Zwick & Velicer, 1982), the reliability of the factors (Cliff, 1988, 1992), or on the MV-to-factor ratio and the range of communalities (Tucker, Koopman, & Linn,1969). Thus, the general conclusion is that there is little justification for using the Kaiser criterion to decide how many factors to retain. ... There is little theoretical evidence to support it, ample evidence to the contrary, and better alternatives that were ignored."
answered Dec 29, 2016 at 23:55
-
1$\begingroup$ S. Mulaik in his monography "Foundations of factor analysis" gave as baseline the simple argument: since we want to find common factors/components, where each explains more than one variable, each factor should collect variance of more than value of 1 . After that, I'll second Preacher/MacCallums' thoughts on this subject - that's a really good article! $\endgroup$ Jan 2, 2017 at 15:39