6
$\begingroup$

I am trying following example of structural equation model in R:

    require(lavaan)
    HS.model <- " visual  =~ x1 + x2 + x3      
            +               textual =~ x4 + x5 + x6
            +               speed   =~ x7 + x8 + x9 "
            > 
    fit <- cfa(HS.model, data = HolzingerSwineford1939)
    summary(fit, fit.measures=TRUE, modindices=T)
    lavaan (0.5-18) converged normally after  35 iterations

      Number of observations                           301

      Estimator                                         ML
      Minimum Function Test Statistic               85.306
      Degrees of freedom                                24
      P-value (Chi-square)                           0.000

    Model test baseline model:

      Minimum Function Test Statistic              918.852
      Degrees of freedom                                36
      P-value                                        0.000

    User model versus baseline model:

      Comparative Fit Index (CFI)                    0.931
      Tucker-Lewis Index (TLI)                       0.896

    Loglikelihood and Information Criteria:

      Loglikelihood user model (H0)              -3737.745
      Loglikelihood unrestricted model (H1)      -3695.092

      Number of free parameters                         21
      Akaike (AIC)                                7517.490
      Bayesian (BIC)                              7595.339
      Sample-size adjusted Bayesian (BIC)         7528.739

    Root Mean Square Error of Approximation:

      RMSEA                                          0.092
      90 Percent Confidence Interval          0.071  0.114
      P-value RMSEA <= 0.05                          0.001

    Standardized Root Mean Square Residual:

      SRMR                                           0.065

    Parameter estimates:

      Information                                 Expected
      Standard Errors                             Standard

                       Estimate  Std.err  Z-value  P(>|z|)
    Latent variables:
      visual =~
        x1                1.000
        x2                0.554    0.100    5.554    0.000
        x3                0.729    0.109    6.685    0.000
      textual =~
        x4                1.000
        x5                1.113    0.065   17.014    0.000
        x6                0.926    0.055   16.703    0.000
      speed =~
        x7                1.000
        x8                1.180    0.165    7.152    0.000
        x9                1.082    0.151    7.155    0.000

    Covariances:
      visual ~~
        textual           0.408    0.074    5.552    0.000
        speed             0.262    0.056    4.660    0.000
      textual ~~
        speed             0.173    0.049    3.518    0.000

    Variances:
        x1                0.549    0.114
        x2                1.134    0.102
        x3                0.844    0.091
        x4                0.371    0.048
        x5                0.446    0.058
        x6                0.356    0.043
        x7                0.799    0.081
        x8                0.488    0.074
        x9                0.566    0.071
        visual            0.809    0.145
        textual           0.979    0.112
        speed             0.384    0.086

The model appears good from goodness of fit indices. The modification indices are following:

    Modification Indices:

           lhs op     rhs     mi    epc sepc.lv sepc.all sepc.nox
    1   visual =~      x1     NA     NA      NA       NA       NA
    2   visual =~      x2  0.000  0.000   0.000    0.000    0.000
    3   visual =~      x3  0.000  0.000   0.000    0.000    0.000
    4  textual =~      x4     NA     NA      NA       NA       NA
    5  textual =~      x5  0.000  0.000   0.000    0.000    0.000
    6  textual =~      x6  0.000  0.000   0.000    0.000    0.000
    7    speed =~      x7     NA     NA      NA       NA       NA
    8    speed =~      x8  0.000  0.000   0.000    0.000    0.000
    9    speed =~      x9  0.000  0.000   0.000    0.000    0.000
    10      x1 ~~      x1  0.000  0.000   0.000    0.000    0.000
    11      x2 ~~      x2  0.000  0.000   0.000    0.000    0.000
    12      x3 ~~      x3  0.000  0.000   0.000    0.000    0.000
    13      x4 ~~      x4  0.000  0.000   0.000    0.000    0.000
    14      x5 ~~      x5  0.000  0.000   0.000    0.000    0.000
    15      x6 ~~      x6  0.000  0.000   0.000    0.000    0.000
    16      x7 ~~      x7  0.000  0.000   0.000    0.000    0.000
    17      x8 ~~      x8  0.000  0.000   0.000    0.000    0.000
    18      x9 ~~      x9  0.000  0.000   0.000    0.000    0.000
    19  visual ~~  visual  0.000  0.000   0.000    0.000    0.000
    20 textual ~~ textual  0.000  0.000   0.000    0.000    0.000
    21   speed ~~   speed  0.000  0.000   0.000    0.000    0.000
    22  visual ~~ textual  0.000  0.000   0.000    0.000    0.000
    23  visual ~~   speed  0.000  0.000   0.000    0.000    0.000
    24 textual ~~   speed  0.000  0.000   0.000    0.000    0.000
    25  visual =~      x4  1.211  0.077   0.069    0.059    0.059
    26  visual =~      x5  7.441 -0.210  -0.189   -0.147   -0.147
    27  visual =~      x6  2.843  0.111   0.100    0.092    0.092
    28  visual =~      x7 18.631 -0.422  -0.380   -0.349   -0.349
    29  visual =~      x8  4.295 -0.210  -0.189   -0.187   -0.187
    30  visual =~      x9 36.411  0.577   0.519    0.515    0.515
    31 textual =~      x1  8.903  0.350   0.347    0.297    0.297
    32 textual =~      x2  0.017 -0.011  -0.011   -0.010   -0.010
    33 textual =~      x3  9.151 -0.272  -0.269   -0.238   -0.238
    34 textual =~      x7  0.098 -0.021  -0.021   -0.019   -0.019
    35 textual =~      x8  3.359 -0.121  -0.120   -0.118   -0.118
    36 textual =~      x9  4.796  0.138   0.137    0.136    0.136
    37   speed =~      x1  0.014  0.024   0.015    0.013    0.013
    38   speed =~      x2  1.580 -0.198  -0.123   -0.105   -0.105
    39   speed =~      x3  0.716  0.136   0.084    0.075    0.075
    40   speed =~      x4  0.003 -0.005  -0.003   -0.003   -0.003
    41   speed =~      x5  0.201 -0.044  -0.027   -0.021   -0.021
    42   speed =~      x6  0.273  0.044   0.027    0.025    0.025
    43      x1 ~~      x2  3.606 -0.184  -0.184   -0.134   -0.134
    44      x1 ~~      x3  0.935 -0.139  -0.139   -0.105   -0.105
    45      x1 ~~      x4  3.554  0.078   0.078    0.058    0.058
    46      x1 ~~      x5  0.522 -0.033  -0.033   -0.022   -0.022
    47      x1 ~~      x6  0.048  0.009   0.009    0.007    0.007
    48      x1 ~~      x7  5.420 -0.129  -0.129   -0.102   -0.102
    49      x1 ~~      x8  0.634 -0.041  -0.041   -0.035   -0.035
    50      x1 ~~      x9  7.335  0.138   0.138    0.117    0.117
    51      x2 ~~      x3  8.532  0.218   0.218    0.164    0.164
    52      x2 ~~      x4  0.534 -0.034  -0.034   -0.025   -0.025
    53      x2 ~~      x5  0.023 -0.008  -0.008   -0.005   -0.005
    54      x2 ~~      x6  0.785  0.039   0.039    0.031    0.031
    55      x2 ~~      x7  8.918 -0.183  -0.183   -0.143   -0.143
    56      x2 ~~      x8  0.054 -0.012  -0.012   -0.010   -0.010
    57      x2 ~~      x9  1.895  0.075   0.075    0.063    0.063
    58      x3 ~~      x4  0.142 -0.016  -0.016   -0.012   -0.012
    59      x3 ~~      x5  7.858 -0.130  -0.130   -0.089   -0.089
    60      x3 ~~      x6  1.855  0.055   0.055    0.044    0.044
    61      x3 ~~      x7  0.638 -0.044  -0.044   -0.036   -0.036
    62      x3 ~~      x8  0.059 -0.012  -0.012   -0.011   -0.011
    63      x3 ~~      x9  4.126  0.102   0.102    0.089    0.089
    64      x4 ~~      x5  2.534  0.186   0.186    0.124    0.124
    65      x4 ~~      x6  6.220 -0.235  -0.235   -0.185   -0.185
    66      x4 ~~      x7  5.920  0.098   0.098    0.078    0.078
    67      x4 ~~      x8  3.805 -0.069  -0.069   -0.059   -0.059
    68      x4 ~~      x9  0.196 -0.016  -0.016   -0.014   -0.014
    69      x5 ~~      x6  0.916  0.101   0.101    0.072    0.072
    70      x5 ~~      x7  1.233 -0.049  -0.049   -0.035   -0.035
    71      x5 ~~      x8  0.347  0.023   0.023    0.018    0.018
    72      x5 ~~      x9  0.999  0.040   0.040    0.031    0.031
    73      x6 ~~      x7  0.259 -0.020  -0.020   -0.017   -0.017
    74      x6 ~~      x8  0.275  0.018   0.018    0.016    0.016
    75      x6 ~~      x9  0.097 -0.011  -0.011   -0.010   -0.010
    76      x7 ~~      x8 34.145  0.536   0.536    0.488    0.488
    77      x7 ~~      x9  5.183 -0.187  -0.187   -0.170   -0.170
    78      x8 ~~      x9 14.946 -0.423  -0.423   -0.415   -0.415

How do I interpret these modification indices? Are there any pointers to indicate how the model can be improved? Thanks for your insight.

$\endgroup$
7
$\begingroup$

In most CFA analyses, the chi sq value will not reach p>.05, especially if you have a large N. Most people look for CMIN, i.e., chisq/df, of <3, or the change in chi sq between nested models, i.e., two models with a minor change in structure, the chi sq for this being (diff in chisq) with (diff in df) df.

The Modification Indices suggest links to change in your structure. Do this incrementally, checking the change in chi sq after each one, to see if it has really helped. You should only make changes that are theoretically sensible, in terms of your model. Start with the largest sensible modofication.

The MIs with the =~ operator are most use, as these are between latent (endogenous) and observed (exogenous) variables.

Once you have exhausted these, the ~~ operators indicate additional links between factors, or error variances. Be careful here. Most analysts accept adding links between error variances for observed variables that form the same latent variable, or for observed variables that have some relationship not captured by the latent variables in the model (e.g., measurement method, perhaps).

$\endgroup$
  • 1
    $\begingroup$ It can help to print out the modification indices in sorted order. Lavaan does this through modificationindices(fit, sort. = TRUE). Note the period after sort. $\endgroup$ – Placidia Jan 18 '16 at 14:58
4
$\begingroup$

The model is actually not that good. See, you have a chi-square of 85.306 with p-value of 0.000. You want it to be not statistically significant instead. Also, the other fit indices are easier to interpret when comparing two different models. By themselves they mean little to nothing. Some authors have suggested that RMSEA below .08 can be used to argue for good fit, but that is not agreed upon in the field and your RMSEA is above that threshold.

Improving your model by fiddling with MI

Some authors do not recommend trying to improve your model by fiddling with MI scores. However, AFAIK the bigger the mi value, the most influential is the variable.

You can spot those variables by running:

mod_ind <- modificationindices(fit)

Spoting the top 10:

head(mod_ind[order(mod_ind$mi, decreasing=TRUE), ], 10)

And the bigger than 5:

subset(mod_ind[order(mod_ind$mi, decreasing=TRUE), ], mi > 5)

Again, this should only be done if you have a hypothesis for what might be going on in the data.

Here is the reference to the code: http://jeromyanglim.tumblr.com/post/33556941601/lavaan-cheat-sheet

$\endgroup$
  • 1
    $\begingroup$ It seems like you imply that a significant chi-square (i.e. p = 0.000 here) indicates a bad model. That's not necessarily true, as it's very sensitive to sample size. I would rather state that the model could probably be improved, as the TLI, CFI, RMSEA and model's Chi-square + p-value are all not meeting / exceeding cut-offs (see e.g.: tandfonline.com/doi/abs/10.1080/…). And this is confirmed when inspecting the modification indices, as there are several really high ones (e.g.,: x7 ~~ x8: 30+) $\endgroup$ – Amonet Mar 31 '18 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.