1
$\begingroup$

I am conducting an EFA (exploratory factor analysis) in SAS using maximum likelihood estimation (ML). I am trying to understand the eigenvalues output and how they should be used in interpreting the number of factors to extract (via the scree plot or the Kaiser criterion of eigenvalues > 1).

First, my communalities estimated are presented below. 

                          Prior Communality Estimates: SMC    

  Math1         Math2         Math3         Math4          Par1          Par2          Par3

0.70383529    0.67370127    0.69961121    0.39254595    0.45469663    0.40570330    0.57243489

     Par4            Par5            Eng1            Eng2            Eng3            Eng4

  0.40805226      0.47669350      0.53707305      0.58089883      0.60824851      0.44739130

The sum of my communalities is 6.960886. In other methods, the sum of the communalities is partitioned by the factors to get the preliminary eigenvalues extracted from the data. This is not what happens in SAS's maximum likelihood. Instead, the total value partitioned is 16.7804663. See below.

Preliminary Eigenvalues: Total = 16.7804663  Average = 1.2908051


                      Eigenvalue    Difference    Proportion    Cumulative

                 1    9.07979678    3.65235893        0.5411        0.5411
                 2    5.42743785    1.77049075        0.3234        0.8645
                 3    3.65694710    2.99021550        0.2179        1.0825
                 4    0.66673160    0.71076308        0.0397        1.1222
                 5    -.04403148    0.07803164       -0.0026        1.1196
                 6    -.12206312    0.01122060       -0.0073        1.1123
                 7    -.13328372    0.03628945       -0.0079        1.1044
                 8    -.16957317    0.05296247       -0.0101        1.0942
                 9    -.22253564    0.06265504       -0.0133        1.0810
                10    -.28519068    0.03336312       -0.0170        1.0640
                11    -.31855380    0.00849750       -0.0190        1.0450
                12    -.32705130    0.10111286       -0.0195        1.0255
                13    -.42816416                     -0.0255        1.0000

Now compare the above estimation of the initial eigenvalues using ML extraction to that which is done when ULS extraction is used. Notice the sum of the communalities is equal to the total variance being partitioned by the factors in computing the eigenvalues.

                          Prior Communality Estimates: SMC    

  Math1         Math2         Math3         Math4          Par1          Par2          Par3

0.70383529    0.67370127    0.69961121    0.39254595    0.45469663    0.40570330    0.57243489

     Par4            Par5            Eng1            Eng2            Eng3            Eng4

  0.40805226      0.47669350      0.53707305      0.58089883      0.60824851      0.44739130


            Preliminary Eigenvalues: Total = 6.96088599  Average = 0.53545277

                      Eigenvalue    Difference    Proportion    Cumulative

                 1    3.62496506    1.46659947        0.5208        0.5208
                 2    2.15836559    0.42687464        0.3101        0.8308
                 3    1.73149094    1.36990342        0.2487        1.0796
                 4    0.36158752    0.38266012        0.0519        1.1315
                 5    -.02107260    0.03255722       -0.0030        1.1285
                 6    -.05362983    0.00897655       -0.0077        1.1208
                 7    -.06260638    0.01262491       -0.0090        1.1118
                 8    -.07523129    0.02541048       -0.0108        1.1010
                 9    -.10064178    0.02409404       -0.0145        1.0865
                10    -.12473581    0.02150386       -0.0179        1.0686
                11    -.14623967    0.01648174       -0.0210        1.0476
                12    -.16272141    0.00592293       -0.0234        1.0242
                13    -.16864434                     -0.0242        1.0000

Questions:

  1. How is ML extraction computing the total estimate of variance to be partitioned? (I've searched for documentation but have had no luck.)
  2. Should the preliminary eigenvalues output from SAS be used in evaluating the Kaiser criterion and the scree plot? (All the examples I've found do indeed use these estimates.)
  3. If I use the preliminary eigenvalues in evaluating the extraction criterion, is this really comparable to what would be done with the other methods or in other programs (see SPSS output below)? If SAS ML eigenvalues are scaled in some way, then wouldn't you be more likely to retain more factors using them?

Some additional observations: When I divide the ML estimated eigenvalues by the ratio of the total eigenvalues to the sum of the communalities (i.e., 6.960886/16.78047), I get estimates that are similar to the ULS produced eigenvalue estimates. Is some kind of scaling factor used?

Factor  ML Estimate     Divided by Ratio        ULS Estimate
1       9.07979678      3.766488322             3.62496506
2       5.42743785      2.251413966             2.15836559
3       3.6569471       1.516977624             1.73149094
4       0.6667316       0.276574118             0.36158752
5       -0.04403148     -0.018265173            -0.0210726
6       -0.12206312     -0.050634318            -0.05362983
7       -0.13328372     -0.055288856            -0.06260638
8       -0.16957317     -0.070342474            -0.07523129
9       -0.22253564     -0.092312406            -0.10064178
10      -0.28519068     -0.118303018            -0.12473581
11      -0.3185538      -0.132142734            -0.14623967
12      -0.3270513      -0.135667673            -0.16272141
13      -0.42816416     -0.177611388            -0.16864434

SAS also outputs eigenvalues of the weighted reduced correlation matrix and, after specifying the number of factors to extract, the variance explained by each factor (both weighted and unweighted). See below for an example. This acknowledges that there is some kind of weighting going on. Notice the unweighted estimates are similar to those produced by ULS.

Eigenvalues of the Weighted Reduced Correlation Matrix: Total = 23.9975741  Average = 1.84596724

                      Eigenvalue    Difference    Proportion    Cumulative

                 1    12.1807402     5.1022541        0.5076        0.5076
                 2     7.0784861     2.3401379        0.2950        0.8025
                 3     4.7383482     4.0140838        0.1975        1.0000
                 4     0.7242644     0.5232741        0.0302        1.0302
                 5     0.2009903     0.1097693        0.0084        1.0386
                 6     0.0912210     0.0819569        0.0038        1.0424
                 7     0.0092641     0.0327097        0.0004        1.0427
                 8    -0.0234456     0.0661096       -0.0010        1.0418
                 9    -0.0895552     0.0686260       -0.0037        1.0380
                10    -0.1581813     0.0432464       -0.0066        1.0314
                11    -0.2014277     0.0396817       -0.0084        1.0230
                12    -0.2411094     0.0709113       -0.0100        1.0130
                13    -0.3120208                     -0.0130        1.0000

                           Variance Explained by Each Factor

                          Factor        Weighted    Unweighted

                          Factor1     12.1807402    3.39853231
                          Factor2      7.0784861    2.44555900
                          Factor3      4.7383482    1.87419868

SPSS outputs the same communalities, the same final eigenvalue estimates as SAS's unweighted estimates, but it produces different initial eigenvalues. See below. So there appears to be some different standards in how the initial eigenvalue estimates are presented.

enter image description here

Improve this question
$\endgroup$
7
  • $\begingroup$ I'm not SAS user and I don't know specific details in the ML factor extraction algorithm, but here is one thing. ML extraction is among methods which weight variables at iterations putting more weight on variables showing more communality. I admit that weighted and unweighted reduced matrices you cite reflect namely that weighting. Next thing, ML extraction is not eigenvalue-seeking, unlike PAF. Its extraction SS or variance which is what SAS probably calls eigenvalues are not the objective function (unlike in PAF) but is by-product. $\endgroup$
    – ttnphns
    Aug 8, 2015 at 20:07
  • $\begingroup$ (cont.) Third thing. Kaiser (eigen>1) or Cattell (scree-plot elbow) rules to decide on the number of factors m in FA are, typically, applied to PCA done prior the FA rather than to the FA itself. So, when you come to FA extraction you normally already have decided how many factors to extract. You cannot decide on m easily within FA process because its results depend on m. In short, FA assumes that you already know m. $\endgroup$
    – ttnphns
    Aug 8, 2015 at 20:13
  • $\begingroup$ @ttnphns - Your comments helped me think about this a little more but I'm not sure I understand you completely. I have edited my question above to try and clarify my questions and better reflect my current understanding. I think I'm still confused about some of the SAS terminology (eigenvalue vs extraction SS) and when to apply the extraction criteria (I thought the eigenvalues produced by PCA were comparable to those produced by EFA with the exception of the communalities upon which they were built: 1 vs squared multiple correlations). Please let me know if you have any additional comments. $\endgroup$
    – ESmith5988
    Aug 9, 2015 at 20:05
  • $\begingroup$ Initial eigenvalues in SPSS output are PCA eigenvalues, i.e. those of the unreduced corr. matrix. You can see yourself: all the values are positive (which only seldom occurs with reduced matrix). Now, ML extraction (requested to extract m=3 factors, be it by Kaiser ">1" rule based on PCA's eigenvalues or by user's direct request) extracted the factors and produced the matrix of loadings. Its column SS are the factor's variances and are displayed on the right. ML extraction is not eigenvalue-based, it produces loadings by directly fitting the correlations. $\endgroup$
    – ttnphns
    Aug 9, 2015 at 22:15
  • 1
    $\begingroup$ Here is an illustration to my words. Get the loading matrix A and compute A*A' which, as we know from the theory of FA, is the reproduced reduced correlation matrix. Compute eigenvalues of it. You'll see that they are not exactly the extraction SS of loadings (although the correlation between these and those is very high). This fact is because MLE extraction is not eigenvalue-based: SS loadings are not eigenvalues (in the usual meaning of the word - as linear roots of the corr matrix). But if you do all the same test with PAF extraction results you'll see that SS loadings = eigenvalues. $\endgroup$
    – ttnphns
    Aug 9, 2015 at 23:08

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.