Skip to main content
Cross Validated

Return to Question

Post Closed as "Not suitable for this site" by Shawn Hemelstrand, User1865345, utobi
added 3304 characters in body
Source Link
zstats
zstats
  • 11
  • 3

UPDATE (2 May 2024): To include the summary statistics for fit.scalar below. The model converged and the model fit statistics seem to suggest an adequate fit (with scaled CFI ~ 0.96, scaled TLI ~ 0.96, scaled RMSEA ~0.065, SRMR ~ 0.055). Could this be a case of sample size issues for chi-square (I have N=699 in total, divided into 345 and 354 for group 1 and group 2)

lavaan 0.6.16 ended normally after 201 iterations

  Estimator                                       DWLS
  Optimization method                           NLMINB
  Number of model parameters                       522
  Number of equality constraints                   208

  Number of observations per group:                   
    Male                                           345
    Female                                         354

Model Test User Model:
                                              Standard      Scaled
  Test Statistic                              3922.985    4517.138
  Degrees of freedom                              1836        1836
  P-value (Chi-square)                           0.000       0.000
  Scaling correction factor                                  1.181
  Shift parameter                                         1196.628
    simple second-order correction                                
  Test statistic for each group:
    Male                                      2181.037    2436.694
    Female                                    1741.948    2080.445

Model Test Baseline Model:

  Test statistic                            516568.414   65863.161
  Degrees of freedom                              1806        1806
  P-value                                        0.000       0.000
  Scaling correction factor                                  8.036

User Model versus Baseline Model:

  Comparative Fit Index (CFI)                    0.996       0.958
  Tucker-Lewis Index (TLI)                       0.996       0.959
                                                                  
  Robust Comparative Fit Index (CFI)                            NA
  Robust Tucker-Lewis Index (TLI)                               NA

Root Mean Square Error of Approximation:

  RMSEA                                          0.057       0.065
  90 Percent confidence interval - lower         0.055       0.062
  90 Percent confidence interval - upper         0.060       0.067
  P-value H_0: RMSEA <= 0.050                    0.000       0.000
  P-value H_0: RMSEA >= 0.080                    0.000       0.000
                                                                  
  Robust RMSEA                                                  NA
  90 Percent confidence interval - lower                        NA
  90 Percent confidence interval - upper                        NA
  P-value H_0: Robust RMSEA <= 0.050                            NA
  P-value H_0: Robust RMSEA >= 0.080                            NA

Standardized Root Mean Square Residual:

  SRMR                                           0.055       0.055

Parameter Estimates:

  Standard errors                           Robust.sem
  Information                                 Expected
  Information saturated (h1) model        Unstructured

UPDATE (2 May 2024): To include the summary statistics for fit.scalar below. The model converged and the model fit statistics seem to suggest an adequate fit (with scaled CFI ~ 0.96, scaled TLI ~ 0.96, scaled RMSEA ~0.065, SRMR ~ 0.055). Could this be a case of sample size issues for chi-square (I have N=699 in total, divided into 345 and 354 for group 1 and group 2)

lavaan 0.6.16 ended normally after 201 iterations

  Estimator                                       DWLS
  Optimization method                           NLMINB
  Number of model parameters                       522
  Number of equality constraints                   208

  Number of observations per group:                   
    Male                                           345
    Female                                         354

Model Test User Model:
                                              Standard      Scaled
  Test Statistic                              3922.985    4517.138
  Degrees of freedom                              1836        1836
  P-value (Chi-square)                           0.000       0.000
  Scaling correction factor                                  1.181
  Shift parameter                                         1196.628
    simple second-order correction                                
  Test statistic for each group:
    Male                                      2181.037    2436.694
    Female                                    1741.948    2080.445

Model Test Baseline Model:

  Test statistic                            516568.414   65863.161
  Degrees of freedom                              1806        1806
  P-value                                        0.000       0.000
  Scaling correction factor                                  8.036

User Model versus Baseline Model:

  Comparative Fit Index (CFI)                    0.996       0.958
  Tucker-Lewis Index (TLI)                       0.996       0.959
                                                                  
  Robust Comparative Fit Index (CFI)                            NA
  Robust Tucker-Lewis Index (TLI)                               NA

Root Mean Square Error of Approximation:

  RMSEA                                          0.057       0.065
  90 Percent confidence interval - lower         0.055       0.062
  90 Percent confidence interval - upper         0.060       0.067
  P-value H_0: RMSEA <= 0.050                    0.000       0.000
  P-value H_0: RMSEA >= 0.080                    0.000       0.000
                                                                  
  Robust RMSEA                                                  NA
  90 Percent confidence interval - lower                        NA
  90 Percent confidence interval - upper                        NA
  P-value H_0: Robust RMSEA <= 0.050                            NA
  P-value H_0: Robust RMSEA >= 0.080                            NA

Standardized Root Mean Square Residual:

  SRMR                                           0.055       0.055

Parameter Estimates:

  Standard errors                           Robust.sem
  Information                                 Expected
  Information saturated (h1) model        Unstructured
edited title
Link
zstats
zstats
  • 11
  • 3

CFA Measurement Invariance - can weak invariance be significant with the configural but not strong invariance?

Source Link
zstats
zstats
  • 11
  • 3

CFA Measurement Invariance - can weak invariance significant with the configural but not strong invariance?

I am a newbie here. I searched through the forum but did not find similar questions, so I hope someone who is proficient with SEM / lavaan can help out.

I am using R lavaan and semTools to perform measurement invariance analysis based on the following models:

fit.original <- lavaan::cfa(model = cfa_model_expr, data = data_df, ordered = TRUE, group = "GENDER")

fit.metric <- lavaan::cfa(model = cfa_model_expr, data = data_df, ordered = TRUE, group = "GENDER", group.equal = c("loadings"))

fit.scalar <- lavaan::cfa(model = cfa_model_expr, data = data_df, ordered = TRUE, group = "GENDER", group.equal = c("loadings", "intercepts"))

Once fitted, I used semTools::compareFit to compare these three models, but I got lavaan warning that some models were not nested or less restricted model.

Warning: lavaan WARNING:
    Some restricted models fit better than less restricted models;
    either these models are not nested, or the less restricted model
    failed to reach a global optimum. Smallest difference =
    -540.550720801662
Scaled Chi-Squared Difference Test (method = “satorra.2000”)

lavaan NOTE:
    The “Chisq” column contains standard test statistics, not the
    robust test that should be reported per model. A robust difference
    test is a function of two standard (not robust) statistics.
 
             Df AIC BIC  Chisq Chisq diff Df diff    Pr(>Chisq)    
m.original 1678         3786.0                                     
m.metric   1714         4463.5     102.06      36 0.00000003094 ***
m.scalar   1836         3923.0    -717.11     122             1    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

So I proceed to use anova to do pairwise comparison:

Scaled Chi-Squared Difference Test (method = “satorra.2000”)

lavaan NOTE:
    The “Chisq” column contains standard test statistics, not the
    robust test that should be reported per model. A robust difference
    test is a function of two standard (not robust) statistics.
 
             Df AIC BIC  Chisq Chisq diff Df diff    Pr(>Chisq)    
m.original 1678         3786.0                                     
m.metric   1714         4463.5     102.06      36 0.00000003094 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Warning: lavaan WARNING:
    Some restricted models fit better than less restricted models;
    either these models are not nested, or the less restricted model
    failed to reach a global optimum. Smallest difference =
    -540.550720801662
Scaled Chi-Squared Difference Test (method = “satorra.2000”)

lavaan NOTE:
    The “Chisq” column contains standard test statistics, not the
    robust test that should be reported per model. A robust difference
    test is a function of two standard (not robust) statistics.
 
           Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)
m.metric 1714         4463.5                              
m.scalar 1836         3923.0    -717.11     122          1

So the culprit appears to be between m.metric and m.scalar. Then if I compare m.original and m.scalar, I got a non-significant result:

Scaled Chi-Squared Difference Test (method = “satorra.2000”)

lavaan NOTE:
    The “Chisq” column contains standard test statistics, not the
    robust test that should be reported per model. A robust difference
    test is a function of two standard (not robust) statistics.
 
             Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq)  
m.original 1678          3786                                
m.scalar   1836          3923      185.4     158    0.06715 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

It does not make sense here because that means my original model satisfied the factor loadings + intercepts invariance between groups but not factor loadings itself (weak invariance)?