Is the Kaiser rule (choosing only PCs with eigenvalues >1, see e.g. here on Wikipedia) a reasonable method to select the number of principal components to retain? What are its pros and cons?
The advantage of the rule is that it is easy to calculate, especially if you live in the 1950s, and don't have access to a fast computer.
The disadvantages ... well, I'm going to quote Preacher and MacCallum, in their paper "Repairing Tom Swift’s Electric Factor Analysis Machine". It's worth reading the whole paper, available here: http://www.quantpsy.org/pubs/preacher_maccallum_2003.pdf
"... use of the rule in practice is problematic for several reasons. First, Guttman’s proof regarding the weakest lower bound applies to the population correlation matrix and assumes that the model holds exactly in the population with m factors. In practice, of course, the population correlation matrix is not available and the model will not hold exactly. Application of the rule to a sample correlation matrix under conditions of imperfect model fit represents circumstances under which the theoretical foundation of the rule is no longer applicable. Second, the Kaiser criterion is appropriately applied to eigenvalues of the unreduced correlation matrix rather than to those of the reduced correlation matrix. In practice, the criterion is often misapplied to eigenvalues of a reduced correlation matrix. Third, Gorsuch (1983) noted that many researchers interpret the Kaiser criterion as the actual number of factors to retain rather than as a lower bound for the number of factors. In addition, other researchers have found that the criterion underestimates (Cattell & Vogelmann, 1977; Cliff, 1988; Humphreys, 1964) or overestimates (Browne, 1968; Cattell & Vogelmann, 1977; Horn, 1965; Lee & Comrey, 1979; Linn, 1968; Revelle & Rocklin, 1979; Yeomans & Golder, 1982;Zwick & Velicer, 1982) the number of factors that should be retained. It has also been demonstrated that the number of factors suggested by the Kaiser criterion is dependent on the number of variables (Gorsuch, 1983; Yeomans & Golder, 1982; Zwick & Velicer, 1982), the reliability of the factors (Cliff, 1988, 1992), or on the MV-to-factor ratio and the range of communalities (Tucker, Koopman, & Linn,1969). Thus, the general conclusion is that there is little justification for using the Kaiser criterion to decide how many factors to retain. ... There is little theoretical evidence to support it, ample evidence to the contrary, and better alternatives that were ignored."