Why do I get very bad fit indices for a moderation? (including a negative TLI!) / Multilevel Model

Question

I'm testing L1 moderation models using Mplus.

I'm getting satisfactory fit indices for the model including my main predictor, then the model including a second predictor which will serve as moderator.

However, when I regress the same DV on the interaction term (after GROUPMEAN centering both predictors), not only does the log-likelihood deteriorate a lot (-2LL increases), but my fit indices become just impossible:

• CFI = .02
• TLI = -.6 (yes, negative!)
• RMSEA = .7
• SRMR = .3 (within)

On the other hand, the interaction is significant and the conditional effects (Mean +/- 1 SD) support my hypotheses.

Do the fit indices mean that the model is very bad? Or is there another explanation?

Any thoughts about how to fix the issue (or explore it)?

Answer 1

This is indeed a strange case. As Erik says you can look to see if potential paths have been excluded. In Mplus one easy way to do this is with the modindices output option to request modification indices. Usually you would not want to rely on these only to modify a model, but to check for potential errors or oversights in your code it could be useful?

Answer 2

I would need to know more about your model and data to recommend anything besides looking at modification indices (i.e., what @Erik Ruzek and @michael_zyphur suggest you do). Though my guess is that a small sample size is somewhat responsible for your particularly bad fit indices; as personally, I have only found such poor fit when the sample size is very small (e.g., $n = 150$ or less). Also, it should be noted that the Mplus implementation of the TLI does not truncate values to be between 0 and 1, so a negative value is possible (see this document for more information).