5
$\begingroup$

The psych package in R has a fa.parallel function to help determine the number of factors or components. From the documentation:

One way to determine the number of factors or components in a data matrix or a correlation matrix is to examine the “scree" plot of the successive eigenvalues. Sharp breaks in the plot suggest the appropriate number of components or factors to extract. “Parallel" analyis is an alternative technique that compares the scree of factors of the observed data with that of a random data matrix of the same size as the original. fa.parallel.poly does this for tetrachoric or polychoric analyses.

When I run the function I get the following output:

Parallel analysis suggests that the number of factors =  7  
                            and the number of components =  4

What is the difference between a factor and a component?

$\endgroup$
  • 1
    $\begingroup$ possible duplicate of What are the differences between Factor Analysis and Principal Component Analysis $\endgroup$ – rolando2 Mar 14 '13 at 23:35
  • $\begingroup$ There does seem to be a question here that is not answered in the duplicate, @rolando2: FA "rotates" the principal components, and so ought (by rights) to produce exactly as many "factors" as "components." This "parallel analysis" method (with which I am unfamiliar) appears somehow to select a different number of "factors." $\endgroup$ – whuber Mar 15 '13 at 16:30
3
$\begingroup$

It's talking about principal components. First, it finds the eigenvalues of the correlation matrix which it takes as input. Then it decides how many of those values are "reasonably big" by doing simulations and comparing them with the simulated values. Here is the key part of the code:

valuesx <- eigen(rx)$values

and then later on:

pc.test <- which(!(valuesx > values.sim$mean))[1] - 1
        results$nfact <- fa.test
    results$ncomp <- pc.test
    cat("Parallel analysis suggests that ")
    cat("the number of factors = ", fa.test, " and the number of components = ", 
        pc.test, "\n")

The whole function is written in R, so you can read its source code by typing its name in the R terminal.

Here is a presentation which compares factor analysis with PCA and hopefully answers your question (see the last slide in particular):

http://www.stats.ox.ac.uk/~ripley/MultAnal_HT2007/PC-FA.pdf

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ So, if I understand your answer, a factor is an output of factor analysis. A component is simply an output of principal component analysis? $\endgroup$ – Jim Mar 15 '13 at 23:32
  • 1
    $\begingroup$ Yes. I believe the package is using "component" as shorthand for "principal component" in the output that you quoted. $\endgroup$ – Flounderer Mar 16 '13 at 0:18
3
$\begingroup$

You might wish to read Dinno's Gently Clarifying the Application of Horn’s Parallel Analysis to Principal Component Analysis Versus Factor Analysis. Here's a short distillation:

Principal component analysis (PCA) involves the eigen-decomposition of the correlation matrix $\mathbf{R}$ (or less commonly, the covariance matrix $\mathbf{\Sigma}$), to give eigenvectors (which are generally what the substantive interpretation of PCA is about), and eigenvalues, $\mathbf{\Lambda}$ (which are what the empirical retention decisions, like parallel analysis, are based on).

Common factor analysis (FA) involves the eigen-decomposition of the correlation matrix $\mathbf{R}$ with the diagonal elements replaced with the communalities: $\mathbf{C} = \mathbf{R} - \text{diag}(\mathbf{R}^{+})^{+}$, where $\mathbf{R}^{+}$ indicates the generalized inverse (aka Moore-Penrose inverse, or pseudo-inverse) of matrix $\mathbf{R}$, to also give eigenvectors (which are also generally what the substantive interpretation of FA is about), and eigenvalues, $\mathbf{\Lambda}$ (which, as with PCA, are what the empirical retention decisions, like parallel analysis, are based on).

The eigenvalues, $\mathbf{\Lambda} = \{\lambda_{1}, \dots, \lambda_{p}\}$ ($p$ equals the number of variables producing $\mathbf{R}$) are arranged from largest to smallest, and in a PCA based on $\mathbf{R}$ are interpreted as apportioning $p$ units of total variance under an assumption that each observed variable contributes 1 unit to the total variance. When PCA is based on $\mathbf{\Sigma}$, then each eigenvalue, $\lambda$, is interpreted as apportioning $\text{trace}(\mathbf{\Sigma})$ units of total variance under the assumption that each variable contributes the magnitude of its variance to total variance. In FA, the eigenvalues are interpreted as apportioning $< p$ units of common variance; this interpretation is problematic because eigenvalues in FA can be negative and it is difficult to know how to interpret such values either in terms of apportionment, or in terms of variance.

The parallel analysis procedure involves:

  1. Obtaining $\{\lambda_{1}, \dots, \lambda_{p}\}$ for the observed data, $\mathbf{X}$.
  2. Obtaining $\{\lambda^{r}_{1}, \dots, \lambda^{r}_{p}\}$ for uncorrelated (random) data of the same $n$ and $p$ as $\mathbf{X}$.
  3. Repeating step 2 many times, say $k$ number of times.
  4. Averaging each eigenvalue from step 3 over $k$ to produce $\{\overline{\lambda}^{r}_{1}, \dots, \overline{\lambda}^{r}_{p}\}$.
  5. Retaining those $q$ components or common factors where $\lambda_{q} > \overline{\lambda}^{r}_{q}$

Monte Carlo parallel analysis employs a high centile (e.g. the 95$^{\text{th}}$) rather than the mean in step 4.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I upvoted this answer a long time ago (+1), but would like to remark that the procedure for factor analysis used by the psych package is not what is described in this answer. There are several common methods of factor extraction (and psych implements several of them), but as far as I know none of them amounts to an eigen-decomposition of $\mathbf{C} = \mathbf{R} - \text{diag}(\mathbf{R}^{+})^{+}$ as this answer seems to imply. $\endgroup$ – amoeba Jan 26 '15 at 18:14
  • $\begingroup$ @amoeba Gorsuch, R. L. (1983). Factor Analysis 2nd ed. NJ: Lawrence Erlbaum Associates. Of course there are other ways to estimate the communalities (e.g. iterated factors, etc.). Stata, for example, calculates the default "principal factors" using the math I have outlined. $\endgroup$ – Alexis Jan 26 '15 at 18:40
  • $\begingroup$ I have an electronic copy of the first edition of this book. Can you refer me to a specific section please? In any case, I looked up the manual of psych package and am pretty confident that the method you describe (even if it does exist in the literature) is not implemented there. $\endgroup$ – amoeba Jan 26 '15 at 18:46
  • $\begingroup$ @amoeba Dang it, you are right about the Gorsuch reference (it's been a while since I cited for this area... forgot that I cite him for the common factor model, as opposed to the principal factor extraction formula). Will chase proper refs down now... (re: psych, right they don't do 'common factors'/'principal factors') $\endgroup$ – Alexis Jan 27 '15 at 0:29
  • $\begingroup$ @amoeba Ok, the notation I used is an isomorphism of the r^2 in Gorsuch for common factors still tracking down refs... $\endgroup$ – Alexis Jan 27 '15 at 0:54
1
$\begingroup$

Actually there are two lines, one for the pca and the other for the minres procedure (default) unless another is selected. The program uses the fa$values and the eigenvalues fa$e.values. The fa$values are the values from the common factor solution. The fa$values are less than the eigenvalues.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.