# How to interpret modification indices output of structural equation models?

I am trying following example of structural equation model in R:

    require(lavaan)
HS.model <- " visual  =~ x1 + x2 + x3
+               textual =~ x4 + x5 + x6
+               speed   =~ x7 + x8 + x9 "
>
fit <- cfa(HS.model, data = HolzingerSwineford1939)
summary(fit, fit.measures=TRUE, modindices=T)
lavaan (0.5-18) converged normally after  35 iterations

Number of observations                           301

Estimator                                         ML
Minimum Function Test Statistic               85.306
Degrees of freedom                                24
P-value (Chi-square)                           0.000

Model test baseline model:

Minimum Function Test Statistic              918.852
Degrees of freedom                                36
P-value                                        0.000

User model versus baseline model:

Comparative Fit Index (CFI)                    0.931
Tucker-Lewis Index (TLI)                       0.896

Loglikelihood and Information Criteria:

Loglikelihood user model (H0)              -3737.745
Loglikelihood unrestricted model (H1)      -3695.092

Number of free parameters                         21
Akaike (AIC)                                7517.490
Bayesian (BIC)                              7595.339
Sample-size adjusted Bayesian (BIC)         7528.739

Root Mean Square Error of Approximation:

RMSEA                                          0.092
90 Percent Confidence Interval          0.071  0.114
P-value RMSEA <= 0.05                          0.001

Standardized Root Mean Square Residual:

SRMR                                           0.065

Parameter estimates:

Information                                 Expected
Standard Errors                             Standard

Estimate  Std.err  Z-value  P(>|z|)
Latent variables:
visual =~
x1                1.000
x2                0.554    0.100    5.554    0.000
x3                0.729    0.109    6.685    0.000
textual =~
x4                1.000
x5                1.113    0.065   17.014    0.000
x6                0.926    0.055   16.703    0.000
speed =~
x7                1.000
x8                1.180    0.165    7.152    0.000
x9                1.082    0.151    7.155    0.000

Covariances:
visual ~~
textual           0.408    0.074    5.552    0.000
speed             0.262    0.056    4.660    0.000
textual ~~
speed             0.173    0.049    3.518    0.000

Variances:
x1                0.549    0.114
x2                1.134    0.102
x3                0.844    0.091
x4                0.371    0.048
x5                0.446    0.058
x6                0.356    0.043
x7                0.799    0.081
x8                0.488    0.074
x9                0.566    0.071
visual            0.809    0.145
textual           0.979    0.112
speed             0.384    0.086


The model appears good from goodness of fit indices. The modification indices are following:

    Modification Indices:

lhs op     rhs     mi    epc sepc.lv sepc.all sepc.nox
1   visual =~      x1     NA     NA      NA       NA       NA
2   visual =~      x2  0.000  0.000   0.000    0.000    0.000
3   visual =~      x3  0.000  0.000   0.000    0.000    0.000
4  textual =~      x4     NA     NA      NA       NA       NA
5  textual =~      x5  0.000  0.000   0.000    0.000    0.000
6  textual =~      x6  0.000  0.000   0.000    0.000    0.000
7    speed =~      x7     NA     NA      NA       NA       NA
8    speed =~      x8  0.000  0.000   0.000    0.000    0.000
9    speed =~      x9  0.000  0.000   0.000    0.000    0.000
10      x1 ~~      x1  0.000  0.000   0.000    0.000    0.000
11      x2 ~~      x2  0.000  0.000   0.000    0.000    0.000
12      x3 ~~      x3  0.000  0.000   0.000    0.000    0.000
13      x4 ~~      x4  0.000  0.000   0.000    0.000    0.000
14      x5 ~~      x5  0.000  0.000   0.000    0.000    0.000
15      x6 ~~      x6  0.000  0.000   0.000    0.000    0.000
16      x7 ~~      x7  0.000  0.000   0.000    0.000    0.000
17      x8 ~~      x8  0.000  0.000   0.000    0.000    0.000
18      x9 ~~      x9  0.000  0.000   0.000    0.000    0.000
19  visual ~~  visual  0.000  0.000   0.000    0.000    0.000
20 textual ~~ textual  0.000  0.000   0.000    0.000    0.000
21   speed ~~   speed  0.000  0.000   0.000    0.000    0.000
22  visual ~~ textual  0.000  0.000   0.000    0.000    0.000
23  visual ~~   speed  0.000  0.000   0.000    0.000    0.000
24 textual ~~   speed  0.000  0.000   0.000    0.000    0.000
25  visual =~      x4  1.211  0.077   0.069    0.059    0.059
26  visual =~      x5  7.441 -0.210  -0.189   -0.147   -0.147
27  visual =~      x6  2.843  0.111   0.100    0.092    0.092
28  visual =~      x7 18.631 -0.422  -0.380   -0.349   -0.349
29  visual =~      x8  4.295 -0.210  -0.189   -0.187   -0.187
30  visual =~      x9 36.411  0.577   0.519    0.515    0.515
31 textual =~      x1  8.903  0.350   0.347    0.297    0.297
32 textual =~      x2  0.017 -0.011  -0.011   -0.010   -0.010
33 textual =~      x3  9.151 -0.272  -0.269   -0.238   -0.238
34 textual =~      x7  0.098 -0.021  -0.021   -0.019   -0.019
35 textual =~      x8  3.359 -0.121  -0.120   -0.118   -0.118
36 textual =~      x9  4.796  0.138   0.137    0.136    0.136
37   speed =~      x1  0.014  0.024   0.015    0.013    0.013
38   speed =~      x2  1.580 -0.198  -0.123   -0.105   -0.105
39   speed =~      x3  0.716  0.136   0.084    0.075    0.075
40   speed =~      x4  0.003 -0.005  -0.003   -0.003   -0.003
41   speed =~      x5  0.201 -0.044  -0.027   -0.021   -0.021
42   speed =~      x6  0.273  0.044   0.027    0.025    0.025
43      x1 ~~      x2  3.606 -0.184  -0.184   -0.134   -0.134
44      x1 ~~      x3  0.935 -0.139  -0.139   -0.105   -0.105
45      x1 ~~      x4  3.554  0.078   0.078    0.058    0.058
46      x1 ~~      x5  0.522 -0.033  -0.033   -0.022   -0.022
47      x1 ~~      x6  0.048  0.009   0.009    0.007    0.007
48      x1 ~~      x7  5.420 -0.129  -0.129   -0.102   -0.102
49      x1 ~~      x8  0.634 -0.041  -0.041   -0.035   -0.035
50      x1 ~~      x9  7.335  0.138   0.138    0.117    0.117
51      x2 ~~      x3  8.532  0.218   0.218    0.164    0.164
52      x2 ~~      x4  0.534 -0.034  -0.034   -0.025   -0.025
53      x2 ~~      x5  0.023 -0.008  -0.008   -0.005   -0.005
54      x2 ~~      x6  0.785  0.039   0.039    0.031    0.031
55      x2 ~~      x7  8.918 -0.183  -0.183   -0.143   -0.143
56      x2 ~~      x8  0.054 -0.012  -0.012   -0.010   -0.010
57      x2 ~~      x9  1.895  0.075   0.075    0.063    0.063
58      x3 ~~      x4  0.142 -0.016  -0.016   -0.012   -0.012
59      x3 ~~      x5  7.858 -0.130  -0.130   -0.089   -0.089
60      x3 ~~      x6  1.855  0.055   0.055    0.044    0.044
61      x3 ~~      x7  0.638 -0.044  -0.044   -0.036   -0.036
62      x3 ~~      x8  0.059 -0.012  -0.012   -0.011   -0.011
63      x3 ~~      x9  4.126  0.102   0.102    0.089    0.089
64      x4 ~~      x5  2.534  0.186   0.186    0.124    0.124
65      x4 ~~      x6  6.220 -0.235  -0.235   -0.185   -0.185
66      x4 ~~      x7  5.920  0.098   0.098    0.078    0.078
67      x4 ~~      x8  3.805 -0.069  -0.069   -0.059   -0.059
68      x4 ~~      x9  0.196 -0.016  -0.016   -0.014   -0.014
69      x5 ~~      x6  0.916  0.101   0.101    0.072    0.072
70      x5 ~~      x7  1.233 -0.049  -0.049   -0.035   -0.035
71      x5 ~~      x8  0.347  0.023   0.023    0.018    0.018
72      x5 ~~      x9  0.999  0.040   0.040    0.031    0.031
73      x6 ~~      x7  0.259 -0.020  -0.020   -0.017   -0.017
74      x6 ~~      x8  0.275  0.018   0.018    0.016    0.016
75      x6 ~~      x9  0.097 -0.011  -0.011   -0.010   -0.010
76      x7 ~~      x8 34.145  0.536   0.536    0.488    0.488
77      x7 ~~      x9  5.183 -0.187  -0.187   -0.170   -0.170
78      x8 ~~      x9 14.946 -0.423  -0.423   -0.415   -0.415


How do I interpret these modification indices? Are there any pointers to indicate how the model can be improved? Thanks for your insight.

In most CFA analyses, the chi sq value will not reach p>.05, especially if you have a large N. Most people look for CMIN, i.e., chisq/df, of <3, or the change in chi sq between nested models, i.e., two models with a minor change in structure, the chi sq for this being (diff in chisq) with (diff in df) df.

The Modification Indices suggest links to change in your structure. Do this incrementally, checking the change in chi sq after each one, to see if it has really helped. You should only make changes that are theoretically sensible, in terms of your model. Start with the largest sensible modofication.

The MIs with the =~ operator are most use, as these are between latent (endogenous) and observed (exogenous) variables.

Once you have exhausted these, the ~~ operators indicate additional links between factors, or error variances. Be careful here. Most analysts accept adding links between error variances for observed variables that form the same latent variable, or for observed variables that have some relationship not captured by the latent variables in the model (e.g., measurement method, perhaps).

• It can help to print out the modification indices in sorted order. Lavaan does this through modificationindices(fit, sort. = TRUE). Note the period after sort. – Placidia Jan 18 '16 at 14:58

The model is actually not that good. See, you have a chi-square of 85.306 with p-value of 0.000. You want it to be not statistically significant instead. Also, the other fit indices are easier to interpret when comparing two different models. By themselves they mean little to nothing. Some authors have suggested that RMSEA below .08 can be used to argue for good fit, but that is not agreed upon in the field and your RMSEA is above that threshold.

### Improving your model by fiddling with MI

Some authors do not recommend trying to improve your model by fiddling with MI scores. However, AFAIK the bigger the mi value, the most influential is the variable.

You can spot those variables by running:

mod_ind <- modificationindices(fit)


Spoting the top 10:

head(mod_ind[order(mod_ind$mi, decreasing=TRUE), ], 10)  And the bigger than 5: subset(mod_ind[order(mod_ind$mi, decreasing=TRUE), ], mi > 5)


Again, this should only be done if you have a hypothesis for what might be going on in the data.

Here is the reference to the code: http://jeromyanglim.tumblr.com/post/33556941601/lavaan-cheat-sheet

• It seems like you imply that a significant chi-square (i.e. p = 0.000 here) indicates a bad model. That's not necessarily true, as it's very sensitive to sample size. I would rather state that the model could probably be improved, as the TLI, CFI, RMSEA and model's Chi-square + p-value are all not meeting / exceeding cut-offs (see e.g.: tandfonline.com/doi/abs/10.1080/…). And this is confirmed when inspecting the modification indices, as there are several really high ones (e.g.,: x7 ~~ x8: 30+) – Amonet Mar 31 '18 at 17:07