CFA: Fixing the loading of the observed variables on the error factors to one vs. fixing the variance of the error terms to one

Question

In most (all?) structural equation modelling software the default to scaling the error variables is to fix the loadings of the observed variables on the error variables to unity and freely estimate the error variances. In contrast, the scale (i.e., variance) of the exogenous latent variable is often fixed to one. I wonder why the same logic isn't applied to the error variables so that their variance is fixed to unity (implying an identity matrix of the error terms) and the loading of the observed variables on the error variables is freely estimated. Is there a mathematical reason for that?

Answer 1

Yes. The reason is that the model isn't identified. Even though you've added constraints (more constraints than should be necessary), and so you have positive degrees of freedom, the individual parameters are not identified.

An example:

First, create some data:

library(dplyr)
library(lavaan)

set.seed(42)

d <- data.frame(F = rnorm(1000)) %>%
  dplyr::mutate(
    y1 = rnorm(1000) + F,
    y2 = rnorm(1000) + F,
    y3 = rnorm(1000) + F,
    y4 = rnorm(1000) + F
  )

Here's the regular CFA, with the default identification of the variance of the latent variable from constraining the first loading to 1.

cfa_model <- "
  f =~ y1 + y2 + y3 + y4
"
cfa_fit_1 <- lavaan::sem(cfa_model, d)

summary(cfa_fit1, standardized = TRUE)

This gives a chi-square of 0.391, with 2 df.

We can also free the loadings and constrain the variance of the latent variable.

cfa_model_2 <- "
  f =~ NA * y1 + y2 + y3 + y4
  f ~~ 1 * f
"
cfa_fit_2 <- lavaan::sem(cfa_model_2, d)

Chi-square is the same as the first model: 0.391

We could constrain all loadings to be equal, by constraining them to 1 (or equivalently, constraing them to equality and fixing the variance of the latent variable to 1).

cfa_model_3 <- "
  f =~ 1 * y1 + 1 * y2 + 1 * y3 + 1 * y4
"
cfa_fit_3 <- lavaan::sem(cfa_model_3, d)
summary(cfa_fit_3, standardized = TRUE)

Chi-square is higher now, at 4.412, and we have 5 df - which we expect, we had 2 df in the previous model, and we've added three constraints.

So instead of fixing the loadings, let's fix the error variances to be equal and free everything else.

cfa_model_4 <- "
  f =~ NA * y1 + y2 + y3 + y4
  y1 ~~ 1 * y1
  y2 ~~ 1 * y2
  y3 ~~ 1 * y3
  y4 ~~ 1 * y4
  
"
cfa_fit_4 <- lavaan::sem(cfa_model_4, d)
summary(cfa_fit_4, standardized = TRUE)

When we run that, we get 5 df, which we expect. But we also get a chi-square of 7.654, which is higher than we expect. But the model is also not identified. Specifically, the variance of the latent variable isn't identified.

Variances:
                   Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
    f                 1.044       NA                      1.000    1.000

This is a local identification problem. The model implied correlation between (say) y1 and y2 is equal to:

loading_1 * f_variance * loading_2

We haven't constrained any of those values, so if it wants to achieve an implied correlation of 0.49, it could do it with:

0.7 * 1.0 * 0.7

OR

0.7 * 0.7 * 1.0

There's no way for the model to distinguish between these (or an infinite number of other models) so it cannot converge upon a solution.

In comments below, @denominator suggested that we could add a constraint to the variance of the latent variable in model 4, and make the model work.

This works, but it only works because the variances were very close to 1.0 anyway, with different data this fails.

d2 <- data.frame(F = rnorm(1000)) %>%
  dplyr::mutate(
    y1 = rnorm(1000) * 5 + F,
    y2 = rnorm(1000) * 5  + F,
    y3 = rnorm(1000) * 5  + F,
    y4 = rnorm(1000) * 5  + F
  )

cfa_model_4 <- " f =~ NA * y1 + y2 + y3 + y4 y1 ~~ 1 * y1 y2 ~~ 1 * y2 y3 ~~ 1 * y3 y4 ~~ 1 * y4 f ~~ 1 * f " cfa_fit_4 <- lavaan::sem(cfa_model_4, d2) summary(cfa_fit_4, standardized = TRUE)

Gives me a chi-square of around 62,000.

Where fitting model 2:

```cfa_model_2 <- "
  f =~ NA * y1 + y2 + y3 + y4
  f ~~ 1 * f
"
cfa_fit_2 <- lavaan::sem(cfa_model_2, d2)
summary(cfa_fit_2, standardized = TRUE)

Gives a chi-square of 0.986 (with 2 df).