# Mathematics behind factor loading in Confirmatory factor analysis/ Structural Equation Modeling

I'm curious about how are the loading in a simple confirmatory factor analysis determined mathematically. Also, the intuition is also hazy to me as to how are the factor loads determined when a variable which is not observed is present in the system and still its impact is being estimated on the observed variables (The factor loading)

The basic concept is as follows:

1. Determine what the covariance matrix should be, as a function of the factor model and the unknown loadings.

2. Calculate the observed variance/covariance matrix from the data.

3. Estimate the parameters. Different software uses different algorithms. A conceptually simple choice is a least squares fit. Choose the parameters to make the theoretical matrix close to the observed matrix.

For a simple, one factor model, the covariance of $X_i, X_j$ is $\lambda_i \times \lambda_j$ and the variance of $X_i$ is $\lambda_i^2 + \sigma_i^2$

If I have $p$ variables loading on the factor, that gives me $2p$ parameters from the model and $p(p+1)/2$ observed entries in the covariance matrix. Note that I am fixing the variance of the latent factor to 1. Alternatively, I can fix one of the loadings to 1 and estimate the latent factor variance.

Note that if $p=2$ I have too many parameters and if $p=3$, the model is saturated and I can choose parameters to give a perfect fit. You want at least 4 observed variables to make this interesting.

• The factor loading is determined from the fact that there is covariance between indicators of a latent variable and thus by analysis of the variances of these indicator variables (the values of which is known to us) we try to find out the impact that a factor/latent variable has on a specific indicator (relative to one indicator for which we fix the loading to 1). Is this the correct way to think about it ? @placidia Nov 22 '15 at 17:29
• you almost have it. the covariance between indicators comes from the loadings from the indicators to their common factor. Nov 22 '15 at 18:32
• It's surprisingly straightforward to code up a simple confirmatory factor analysis in Excel, this can help you to understand what is happening. There's a paper here: link.springer.com/content/pdf/10.3758/BF03192739.pdf Nov 22 '15 at 19:20