Are there any theory and tutorial on Diagonally weighted least squares?

Question

I'm interested in learning about the Diagonally Weighted Least Squares (DWLS) method for confirmatory factor analysis (CFA). I'm using lavaan package in R.

I found a couple of papers on DWLS such as

• A Comparison of Diagonal Weighted Least Squares Robust Estimation Techniques for Ordinal Data,

• Maximum Likelihood (ML) and Diagonally Weighted Least Squares (DWLS) Estimation Procedures: A Comparison of Estimation Bias with Ordinal and Multivariate Non-Normal Data .

These papers mention that DWLS may be used in cases where the normality assumptions of the data are not met and/or the data is ordinal. In the toy data that I'm using, the data is not ordinal but may not meet the normality assumptions either.

When I used modeled the data, I get the following results for ML method:

lavaan 0.6-5 ended normally after 58 iterations

  Estimator                                         ML
  Optimization method                           NLMINB
  Number of free parameters                        127
                                                      
                                                  Used       Total
  Number of observations                           168         273
                                                                  
Model Test User Model:
                                                      
  Test statistic                               890.927
  Degrees of freedom                               539
  P-value (Chi-square)                           0.000

Model Test Baseline Model:

  Test statistic                              4180.744
  Degrees of freedom                               630
  P-value                                        0.000

User Model versus Baseline Model:

  Comparative Fit Index (CFI)                    0.901
  Tucker-Lewis Index (TLI)                       0.884

Loglikelihood and Information Criteria:

  Loglikelihood user model (H0)              -6918.778
  Loglikelihood unrestricted model (H1)      -6473.314
                                                      
  Akaike (AIC)                               14091.556
  Bayesian (BIC)                             14488.299
  Sample-size adjusted Bayesian (BIC)        14086.190

Root Mean Square Error of Approximation:

  RMSEA                                          0.062
  90 Percent confidence interval - lower         0.055
  90 Percent confidence interval - upper         0.070
  P-value RMSEA <= 0.05                          0.003

Standardized Root Mean Square Residual:

  SRMR                                           0.057

And using ML with Robust method

lavaan 0.6-5 ended normally after 58 iterations

  Estimator                                         ML
  Optimization method                           NLMINB
  Number of free parameters                        127
                                                      
                                                  Used       Total
  Number of observations                           168         273
                                                                  
Model Test User Model:
                                              Standard      Robust
  Test Statistic                               890.927     841.272
  Degrees of freedom                               539         539
  P-value (Chi-square)                           0.000       0.000
  Scaling correction factor                                  1.059
    for the Yuan-Bentler correction (Mplus variant) 

Model Test Baseline Model:

  Test statistic                              4180.744    3891.337
  Degrees of freedom                               630         630
  P-value                                        0.000       0.000
  Scaling correction factor                                  1.074

User Model versus Baseline Model:

  Comparative Fit Index (CFI)                    0.901       0.907
  Tucker-Lewis Index (TLI)                       0.884       0.892
                                                                  
  Robust Comparative Fit Index (CFI)                         0.909
  Robust Tucker-Lewis Index (TLI)                            0.893

Loglikelihood and Information Criteria:

  Loglikelihood user model (H0)              -6918.778   -6918.778
  Scaling correction factor                                  1.039
      for the MLR correction                                      
  Loglikelihood unrestricted model (H1)      -6473.314   -6473.314
  Scaling correction factor                                  1.055
      for the MLR correction                                      
                                                                  
  Akaike (AIC)                               14091.556   14091.556
  Bayesian (BIC)                             14488.299   14488.299
  Sample-size adjusted Bayesian (BIC)        14086.190   14086.190

Root Mean Square Error of Approximation:

  RMSEA                                          0.062       0.058
  90 Percent confidence interval - lower         0.055       0.050
  90 Percent confidence interval - upper         0.070       0.065
  P-value RMSEA <= 0.05                          0.003       0.043
                                                                  
  Robust RMSEA                                               0.059
  90 Percent confidence interval - lower                     0.052
  90 Percent confidence interval - upper                     0.067

Standardized Root Mean Square Residual:

  SRMR                                           0.057       0.057

Although I used cfa(model, data = data, estimator = "MLR", std.lv = TRUE, std.ov = TRUE, test = "Satorra-Bentler") for robust method, the result shows the the Yuan-Bentler correction. Why?

And finally, the DWLS method:

lavaan 0.6-5 ended normally after 75 iterations

  Estimator                                       DWLS
  Optimization method                           NLMINB
  Number of free parameters                        127
                                                      
                                                  Used       Total
  Number of observations                           168         273
                                                                  
Model Test User Model:
                                                      
  Test statistic                               298.082
  Degrees of freedom                               539
  P-value (Chi-square)                           1.000

Model Test Baseline Model:

  Test statistic                             19920.394
  Degrees of freedom                               630
  P-value                                        0.000

User Model versus Baseline Model:

  Comparative Fit Index (CFI)                    1.000
  Tucker-Lewis Index (TLI)                       1.015

Root Mean Square Error of Approximation:

  RMSEA                                          0.000
  90 Percent confidence interval - lower         0.000
  90 Percent confidence interval - upper         0.000
  P-value RMSEA <= 0.05                          1.000

Standardized Root Mean Square Residual:

  SRMR                                           0.053

So, my questions are:

1. Why Robust method doesn't show Satorra-Bentler method even though I used it in the function. Am I misunderstanding the lavaan package for the robust method?

2. If the data is not normally distributed, is it okay to use DWLS method?

3. Why are the CFI/TLI is 1 and RMSEA is 0. Is this reportable?

I'm very new to cfa, and I'm attempting to learn this using lavaan package and some papers. I hope somebody can point me in the right direction to learn more about CFA and DWLS method.

Answer 1

1. The robust method doesn't show Satorra-Bentler because, by default, when you specify estimator = 'MLR', the Yuan-Bentler test statistic is used. This is why that's appearing in your results. Although this default option seems obtuse, the source for this answer can be found here: https://lavaan.ugent.be/tutorial/est.html

2. From my understanding, yes, you can use DWLS when data is not normally distributed. I say from my understanding because sometimes the CFA literature regarding when to use estimators is sometimes unclear. That said, usually DWLS and ULS estimators are used when performing a CFA with ordinal data and a small sample size (as is your case, n = 168). If you're working with ordinal data, jump right on it. If you're working with continuous data, I'd suggest estimator = 'MLM' in lavaan - an estimator that produces robust estimates and tries to correct for data non-normality.

3. CFI and TLI are can fall below 0 and above 1. So yes, this values can sometimes even surpass 1. Generally, if it surpasses 1 or is 1, you report CFI/TLI = 1.000 - no need to specify what comes after the "dot". As for the RMSEA, it's quite excellent. You should worry if in the results RMSEA = NA, which can indicate that the RMSEA statistic could not be calculated due to lack of degrees of freedom. So, in overall, your results are reportable.