2
$\begingroup$

Assuming a set of data meet all assumptions for EFA and we are doing a factor analysis (with the SMC used to define the shared variance), how do negative eigenvalues appear and what does that say about the input data? I have spent a bit of time refreshing myself on how to calculate an eigenvalue by hand but I am missing a fundamental connection between eigenvalues and variance in EFA.

Below is an example of negative eigenvalues that appear in a SAS EFA tutorial on the UCLA webpage. EFA with negative eigenvalues

The page goes on to state:

Some of the eigenvalues are negative because the matrix is not of full rank. This means that there are probably only four dimensions (corresponding to the four factors whose eigenvalues are greater than zero). Although it is strange to have a negative variance, this happens because the factor analysis is only analyzing the common variance, which is less than the total variance. If we were doing a principal components analysis, we would have had 1’s on the diagonal, which means that all of the variance is being analyzed (which is another way of saying that we are assuming that we have no measurement error), and we would not have negative eigenvalues. In general, it is not uncommon to have negative eigenvalues.

The above seems to suggest this scenario is not uncommon in EFA and it is due to the matrix not being full rank (googling hasn't helped me understand this in the context of EFA). It also suggests this is in part due to a difference in the common variance (EFA only models common variance) vs total variance (not fully sure where this is in the model).

And a follow-up question:

  • Do the proportion and cumulative variance make sense in the above example SAS output or can they be corrected somehow (set all negative eigenvalues to 0 and recalculate the proportion with a new denominator)?
Improve this question
$\endgroup$
7
  • $\begingroup$ The statement "the matrix is not of full rank" appears wrong and misleading. The basic issue is that when you subtract estimates of measurement variance from a covariance matrix, the result is not necessarily a definite matrix: that is, it can have smallish eigenvalues (up to the magnitude of the measurement error, roughly). The mere fact that all the listed eigenvalues are clearly different from zero proves the matrix being analyzed indeed is of full rank. (Rank deficiency is indicated by eigenvalues equal to zero, up to floating point roundoff error.) $\endgroup$
    – whuber ♦
    Commented Oct 26, 2021 at 17:05
  • $\begingroup$ @whuber - part of your explanation is flying over my head. Does this mean the negative eigenvalues correspond to variance associated with measurement error? Is this shared error variance? $\endgroup$
    – ESmyth5988
    Commented Oct 26, 2021 at 19:05
  • 1
    $\begingroup$ Principal axis FA puts estimates of communalities instead of 1s on the diagonal of correlation matrix (or instead of variances on the diagonal of covariance matrix). Since communalities are less than those (they represent only part of variance), they are causes 2 or 3 of non-positive-definiteness. They are too small, generally and most of the time, for the matrix to be p.s.d. $\endgroup$
    – ttnphns
    Commented Oct 27, 2021 at 20:59
  • $\begingroup$ Example of FA of iris dataset, demonstrating negative eigenvalues, is here. Read also. $\endgroup$
    – ttnphns
    Commented Oct 27, 2021 at 21:00
  • 1
    $\begingroup$ I've added a paragraph "Let us look at the final eigenvalues" to here, that would help you understand the "% of". $\endgroup$
    – ttnphns
    Commented Oct 29, 2021 at 4:48

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.