I am building a model with a highly significant interaction. This interaction was one of our main hypotheses. It is clear, however, that the form that the interaction takes does not represent a meaningful change across levels of either variable (one of which is time). The interaction is of 2 quadratic terms (and all lower order terms). Less terms in the model do not fit the data well. Higher order terms do not improve fit. With the quadratic interaction, we see the predicted trends for different levels of covariates begin together for early time, diverge slighlty for middle timings, and then reconverge for late time values. At most, the differences in the ranges of our data are not clinically meaningful. And the convergence at the end of the time range indicates that subjects end up in the same place anyways. Our best explanation for the divergence in the middle time is a change in selection criteria for entry into the study (not well documented, just possibility).

We have categorized the 2 predictors and tested that interaction, so as not to force a functional form. The predicted values for this show some similar trends to the quadratic interaction initially modeled, but is not as rigid or uniform. Additionally, we tested the interaction in a smaller, independently collected sample and found nothing indicating an interaction.

In short, my question is what more do we need to officially write off this interaction? The trouble I see is that this was pretty much what we wanted to test, and we found something statistically significant, yet it appears to not be the greatest fit for the data, nor offer a clinically important interpretation.

  • $\begingroup$ I'm not sure what you want in an answer. If you did the modelling well (& if you're not sure give the detail) you got a good estimate of an effect (though possibly an artefactual one) & found it to be clinically unimportant. That's the story. $\endgroup$ Commented Nov 15, 2013 at 15:18
  • $\begingroup$ We are using a linear mixed model. The interaction is between age at baseline and time of measurement. We are interested in whether younger at baseline do worse at higher follow-up times than older at baseline with less follow-up (i.e. at the same current age, but older at baseline has less time at risk). I have additional concerns about interactions with these variables, but I am currently just curious how to proceed with the interaction. Have we given sufficient proof that it is not a real association, thus can report a simple no-interactions model, in order to have a simpler message? $\endgroup$
    – rjweyant
    Commented Nov 15, 2013 at 15:52
  • 2
    $\begingroup$ Report to whom? For what purpose? If at first you thought it might be important but found it to be negligible, that would seem like a worthwhile thing to report in many circumstances. The only evidence, as opposed to supposition, that it's unreal, & not just unimportant, might come from the smaller sample - did that in fact disconfirm an effect of the size you saw in the larger one, or just fail to find a "significant" effect"? $\endgroup$ Commented Nov 16, 2013 at 17:35


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.