7
$\begingroup$

I thought that the loadings in factor analysis were the correlations between the observed variables and the latent factors. However, when I do factor analysis in R using the psych package, this does not seem to be the case:

    library(psych)
    set.seed(1)
    X <- matrix(rnorm(200), ncol=10)
    fa1 <- fa(X, nfactors=3, rotate="none", scores=TRUE)

    cor(X, fa1$scores)  #correlations between original variables and factor scores
                   MR2         MR1         MR3
     [1,]  0.465509161  0.87299813  0.03241641
     [2,] -0.010609644 -0.32714571  0.64968725
     [3,] -0.219685860  0.47331827 -0.39132195
     [4,] -0.815516983  0.22669390  0.42273446
     [5,] -0.075178935 -0.40431701 -0.69661843
     [6,] -0.204917832  0.07472006  0.05508017
     [7,]  0.240675941  0.13027263  0.23238220
     [8,]  0.756677687 -0.05621205  0.23746738
     [9,]  0.004384459  0.12095273  0.55100943
    [10,]  0.640507568 -0.67810600  0.18597947

    fa1$loadings[1:10, 1:3]
                   MR2         MR1         MR3
     [1,]  0.433925641  0.82218385  0.02717957
     [2,] -0.009889808 -0.30810366  0.54473104
     [3,] -0.204780777  0.44576800 -0.32810435
     [4,] -0.760186392  0.21349881  0.35444221
     [5,] -0.070078250 -0.38078308 -0.58408054
     [6,] -0.191014719  0.07037085  0.04618204
     [7,]  0.224346738  0.12268990  0.19484113
     [8,]  0.705339180 -0.05294013  0.19910480
     [9,]  0.004086985  0.11391248  0.46199451
    [10,]  0.597050885 -0.63863574  0.15593470

    cor(fa1$scores)  # Check that factor scores are uncorrelated
              MR2          MR1           MR3
    MR2  1.000000e+00 4.266996e-16 -1.299606e-16
    MR1  4.266996e-16 1.000000e+00  1.961151e-16
    MR3 -1.299606e-16 1.961151e-16  1.000000e+00

The loadings and correlations are similar, but I expected them to be the same. I tried looking at the source code for fa but had trouble understanding it. Could someone please tell me how the loadings differ from the correlations?

Update: For each factor, the correlations with the observed variables are constant multiples of the loadings:

cor(X, fa1$scores)/fa1$loadings[1:10, 1:3]
           MR2      MR1      MR3
 [1,] 1.072786 1.061804 1.192675
 [2,] 1.072786 1.061804 1.192675
 [3,] 1.072786 1.061804 1.192675
 [4,] 1.072786 1.061804 1.192675
 [5,] 1.072786 1.061804 1.192675
 [6,] 1.072786 1.061804 1.192675
 [7,] 1.072786 1.061804 1.192675
 [8,] 1.072786 1.061804 1.192675
 [9,] 1.072786 1.061804 1.192675
[10,] 1.072786 1.061804 1.192675
$\endgroup$
4
  • $\begingroup$ These constant multiples are a curious effect; have you by any chance managed to clarify since 2011 why it happens? This might depend on the method used for scores extraction. The fa function implements a variety of methods, as far as I know. The default one might not be the best. $\endgroup$ – amoeba Jan 25 '15 at 15:11
  • $\begingroup$ @amoeba Yes I think so. The answer given by ttnphns is correct. I was wrong to expect that the loadings and the sample correlations would be the same. The constant multiples are the reciprocals of the sample standard deviations of the factor score estimates. The sample variances of the factor score estimates are estimates of the proportion of non-error variance in the factor scores. $\endgroup$ – mark999 Jan 27 '15 at 4:07
  • $\begingroup$ Hmmm. There are around five methods of score extraction implemented in the fa function (score='' parameter). I tried them all now, and none resulted in correlations coinciding with loadings... $\endgroup$ – amoeba Jan 27 '15 at 12:25
  • $\begingroup$ @amoeba They're not expected to coincide, because of error (uniqueness). $\endgroup$ – mark999 Jan 28 '15 at 3:16
5
$\begingroup$

I don't know R very well, so I can't track your code. But factor scores (unless the factors are simply principal components) are always approximate: exact scores cannot be computed because the uniqueness value for each case and variable is eternally unobservable. Thus, observed correlations between computed factor scores and the variables only approximate true correlations between factors and variables, the loadings.

$\endgroup$
5
  • $\begingroup$ Thanks for your answer, but I'm not going to vote on it at this stage because I don't know if it's correct. $\endgroup$ – mark999 Jul 9 '11 at 0:54
  • $\begingroup$ @Mark999. It is correct, I dare say. It is very basics of factor analysis theory. After all, I've just replicated your actions - only with other data - in SPSS, and I got the same results as you (including your "constant multiples" finding). $\endgroup$ – ttnphns Jul 9 '11 at 6:27
  • $\begingroup$ P.S. If you want that "constant miltiplier" to be exactly 1 you need to perform PCA instead of factor analysis (as I've already mentioned it, in my answer). $\endgroup$ – ttnphns Jul 9 '11 at 8:23
  • $\begingroup$ Thanks for your comments. Do you know how these loadings (the ones returned by the software) are defined? $\endgroup$ – mark999 Jul 9 '11 at 12:18
  • 2
    $\begingroup$ As for defined, loadings are - as you know - true correlations between factors and variables. Geometrically, a communality of a variable is a projection of the variable's variance onto the "factor subspace", and the loading is a coordinate of this projection on this or that specific factor. As for computationally, loading depends on the method of factor extraction (e.g. principal axes factoring or least residual factoring, etc.) $\endgroup$ – ttnphns Jul 9 '11 at 14:26
0
$\begingroup$

fa() uses the minres factoring method by default fm="minres".

The loadings correspond to the correlations only with the principal components factorization method. You can compute it with principal():

 fa1 <- principal(X, nfactors = 3, rotate = 'none')
 cor(X, fa1$scores)
              PC1         PC2         PC3
 [1,] -0.10920804  0.53177096  0.62089920
 [2,]  0.38040379  0.25737641 -0.61853742
 [3,] -0.63568952 -0.07448425  0.42456182
 [4,] -0.65982013  0.31649913 -0.44502612
 [5,]  0.01177613 -0.74010933  0.10943722
 [6,] -0.23698177  0.22859832  0.21876281
 [7,]  0.22409045  0.43785156  0.36644127
 [8,]  0.69310850  0.26912793  0.47151066
 [9,]  0.15024503  0.65373157 -0.39777599
 [10,]  0.85889193 -0.23091790 -0.02241569
 fa1$loadings[1:10, 1:3]
              PC1         PC2         PC3
 [1,] -0.10920804  0.53177096  0.62089920
 [2,]  0.38040379  0.25737641 -0.61853742
 [3,] -0.63568952 -0.07448425  0.42456182
 [4,] -0.65982013  0.31649913 -0.44502612
 [5,]  0.01177613 -0.74010933  0.10943722
 [6,] -0.23698177  0.22859832  0.21876281
 [7,]  0.22409045  0.43785156  0.36644127
 [8,]  0.69310850  0.26912793  0.47151066
 [9,]  0.15024503  0.65373157 -0.39777599
 [10,]  0.85889193 -0.23091790 -0.02241569
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.