I'm curious about how are the loading in a simple confirmatory factor analysis determined mathematically. Also, the intuition is also hazy to me as to how are the factor loads determined when a variable which is not observed is present in the system and still its impact is being estimated on the observed variables (The factor loading)


The basic concept is as follows:

  1. Determine what the covariance matrix should be, as a function of the factor model and the unknown loadings.

  2. Calculate the observed variance/covariance matrix from the data.

  3. Estimate the parameters. Different software uses different algorithms. A conceptually simple choice is a least squares fit. Choose the parameters to make the theoretical matrix close to the observed matrix.

For a simple, one factor model, the covariance of $X_i, X_j$ is $\lambda_i \times \lambda_j$ and the variance of $X_i$ is $\lambda_i^2 + \sigma_i^2$

If I have $p$ variables loading on the factor, that gives me $2p$ parameters from the model and $p(p+1)/2$ observed entries in the covariance matrix. Note that I am fixing the variance of the latent factor to 1. Alternatively, I can fix one of the loadings to 1 and estimate the latent factor variance.

Note that if $p=2$ I have too many parameters and if $p=3$, the model is saturated and I can choose parameters to give a perfect fit. You want at least 4 observed variables to make this interesting.

  • $\begingroup$ The factor loading is determined from the fact that there is covariance between indicators of a latent variable and thus by analysis of the variances of these indicator variables (the values of which is known to us) we try to find out the impact that a factor/latent variable has on a specific indicator (relative to one indicator for which we fix the loading to 1). Is this the correct way to think about it ? @placidia $\endgroup$ Nov 22 '15 at 17:29
  • $\begingroup$ you almost have it. the covariance between indicators comes from the loadings from the indicators to their common factor. $\endgroup$
    – Placidia
    Nov 22 '15 at 18:32
  • 1
    $\begingroup$ It's surprisingly straightforward to code up a simple confirmatory factor analysis in Excel, this can help you to understand what is happening. There's a paper here: link.springer.com/content/pdf/10.3758/BF03192739.pdf $\endgroup$ Nov 22 '15 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.