Comparing CFA models (AIC)

I wonder whether I can use AIC for model comparison in this scenario:

Model 1:
A = a + b + c
B = d + e + f
C = g + h + i

Model 2:
A = a + b + c
B = d + e + f
(The third factor with his indicators is missing)

The AIC for the second model is much lower. I’m unsure of applicability, because a latent variable and three indicators are missing.

• If you use R with lavaan package then you simple apply anova(fit1, fit2) where fit1 and fit2 are the correspondent models created by lavaan::cfa command. There is no problem if you do not have a latent variable. – Epaminondas Feb 16 '17 at 10:48
• Thank you. I also considered that model 2 is nested in model 1, because parameters are dropped. However, I can’t find literature about comparing models that differ in their number of factors and in the number of indicators. Are you sure, that the models are nested? Further, I get the following warning, when I use the anova()-function: In lavTestLRT(object = <S4 object of class "lavaan">, SB.classic = TRUE, : lavaan WARNING: some models are based on a different set of observed variables – Lilly Feb 16 '17 at 12:07
• I think that I should reconsider my advice. At first glance, since all the terms of the (smaller) model 2 occur in the larger model 1, it looked ok to say that the models are nested, however this is not the case when the two models have different observed variables. Please take a look at this (groups.google.com/forum/#!topic/lavaan/Ya3Ak4AiTkQ) discussion of lavaan google group with subject "lavaan WARNING: some models are based on a different set of observed variables" and the advices therein. – Epaminondas Feb 16 '17 at 19:12
• Thank you for your advise. It seems that I can't easily compare them. – Lilly Feb 20 '17 at 11:47

The AIC is a function of the loglikelihood and loglikelihoods are only comparable across the same data set, i.e., all the same variables and sample. Thus, you can use AIC to compare these non-nested models:

Model 1:
A = a + b + c
B = d + e + f
C = g + h + i

and

Model 2:
A = a + b + e
B = c + d + h
C = f + g + i

because the exact same data set is used for both models.

In your example, the columns of variables differ, making the loglikelihoods incomparable.