Graphing versus further testing interactions

Question

Following the question here, I have come across different (textbook) recommendations.

Some authors suggest that the effect of the moderating variable is best understood in a graphic form.

Others suggest that after graphing the effect, further significance testing should be undertaken to ascertain if the pattern (in the graph) is real or not (unsure if 'real' is the right word here).

My confusion is that if I have found that the interaction of the moderating variable on the the X-Y relationship is significant, why should this further testing be required.

In other words, the graph should be a sufficient explanation of the moderating effect. After all, it does show the pattern and the initial test showed that the pattern is significant.

Are there any best practice in this regard. I understand, of course, that this depends on the context of a study but it always good to know what the common thinking is on this matter.

As always, many, many thanks.....

Answer 1

I believe you are referring to the recommendation to test for "regions" of significance. The idea is to analyze distinct sets of values along the slopes that have been plotted to illustrate a significant interaction. One can then determine where the significant "part" of the interaction starts with reference to a range of specified values of the moderator and predictor.

Someone might educate the both of us, but frankly this strikes me as nonsense, an example of how computational ritual tends to displace judgment. There either is or is not a significant interaction -- it doesn't start & end anywhere. But once you confirm that the interaction is significant, you need to use judgment to identify the values of the predictor and moderator with reference to which the interaction will have practically significant effects.

E.g., if the efficacy of a medical intervention is conditional on (interacts with) the age of the patient, then you should estimate (with appropriate measures of precision, as well, likely signified by confidence intervals) how big the likely difference in efficacy is likely to be within age ranges in which a practitioner might be contemplating the treatment. Depending on how big that effect is within the relevant age range, one might or might not view the interaction as being a factor that effects the treatment decision. As Peter Flom notes, you are likely to be able to illustrate something like that most effectively with a graph; indeed, reporting an interaction only in a table is also a symptom of mechanized & ritualistic analysis, since the outputs in the table are unlikely to make the meaning and practical significance of an interaction clear.

Rather than identify some text that recommends something that seems dopey, I prefer to recommend a text that sets out an approach that strikes me as sensible. Both

Jaccard, J. & Turrisi, R., Interaction Effects in Multiple Regression, (Sage Publications, Thousand Oaks, Calif., 2003); and

Aiken, L.S., West, S.G. & Reno, R.R. Multiple Regression: Testing and Interpreting Interactions, (Sage Publications, Newbury Park, Calif., 1991)

are excellent. They note the "region of significance" technique but both treat it as pretty unimportant relative to using judgment to probe and explicate interactions once one has identified that they exist.

Answer 2

I don't know what you mean by "real or not" - it would help if you quoted the text.

Graphics are often a very good way to understand interactions. They can be hard to understand from just the numbers.