Lavaan Estimator

Question

I did a three-wave study with repeated measured for each wave.I have two questions: I am doing a CFA for a configural model (testing temporal invariance for each variable of my model at 3 points in time).

configural_model <- '
  # Time 1
  Bullying_1 = ~ a1_bul_1 + a1_bul_2 + a1_bul_3 + a1_bul_4 + 
                 a1_bul_5 + a1_bult_6
  
  # Time 2
  Bullying_2 = ~ a2_bul_1 + a2_bul_2 + a1_bul_3 + a2_bul_4 + 
                 a2_bul_5 + a2_bul_6
  
  # Time 3
  Bullying_3 = ~ a3_bul_1 + a3_bul_2 + a3_bul_3 + a1_bul_4 + 
                 a3_bul_5 + a3_bul_6

# Fit the configural model
fit_configural <- cfa(configural_model, data = df, std.lv = TRUE, 
                      estimator = "MLR")

# Summarize the configural model fit
summary(fit_configural, fit.measures = TRUE, standardized = TRUE)

Questions

1. I am using Lavaan, and my data is non-normal (right skewed) and small (90 obs for each time). Should I use MLR or MLM estimator? I ask this because I am getting much better results with MLM, although I think MLR is more used.
2. I have tried testing the configural model with several variables of my model, one at the time, and I always get crappy RMSEA values, even if I get good CFI, TLI or RSMR. I would understand if it was one variable, but with all of them I am finding this weird.

Answer 1

I will go through both of your questions below:

I am using Lavaan, and my data is non-normal (right skewed) and small (90 obs for each time). Should I use MLR or MLM estimator? I ask this because I am getting much better results with MLM, although I think MLR is more used.

I'm not sure either estimator is going to perform well in this scenario. Your sample size is tiny in the context of SEM and whatever results generated from it are likely to greatly under-estimate the true population values. You could instead consider using a Bayesian SEM with blavaan using sensible priors. Though I am only experiences in lavaan/SEM and Bayes separately and do not have experience with using both in blavaan. Sara Depaoli however has a book and course on Bayesian SEM that may be worth checking out.

I have tried testing the configural model with several variables of my model, one at the time, and I always get crappy RMSEA values, even if I get good CFI, TLI or RSMR. I would understand if it was one variable, but with all of them I am finding this weird.

I would forget about global fit if this is an issue and instead look at local fit. After all, its hard to determine where the issue is coming from if you play whack-a-mole with mod indices and other post-hoc methods. This can be done a variety of ways. Individual case residuals (ICRS) and Bollen correlation residuals using lavResiduals in lavaan are both ways of finding out if there is a local problem somewhere that needs addressing. In the case of Bollen correlation residuals, usually values above .10 may be problematic and worth investigating. Then you can perhaps figure out why the global fit indices don't match expectation, though it could also just be a poor model (which you would consequently just report as is if no mis-specification is in place).