3
$\begingroup$

Following the question here, I have come across different (textbook) recommendations.

Some authors suggest that the effect of the moderating variable is best understood in a graphic form.

Others suggest that after graphing the effect, further significance testing should be undertaken to ascertain if the pattern (in the graph) is real or not (unsure if 'real' is the right word here).

My confusion is that if I have found that the interaction of the moderating variable on the the X-Y relationship is significant, why should this further testing be required.

In other words, the graph should be a sufficient explanation of the moderating effect. After all, it does show the pattern and the initial test showed that the pattern is significant.

Are there any best practice in this regard. I understand, of course, that this depends on the context of a study but it always good to know what the common thinking is on this matter.

As always, many, many thanks.....

$\endgroup$
4
$\begingroup$

I believe you are referring to the recommendation to test for "regions" of significance. The idea is to analyze distinct sets of values along the slopes that have been plotted to illustrate a significant interaction. One can then determine where the significant "part" of the interaction starts with reference to a range of specified values of the moderator and predictor.

Someone might educate the both of us, but frankly this strikes me as nonsense, an example of how computational ritual tends to displace judgment. There either is or is not a significant interaction -- it doesn't start & end anywhere. But once you confirm that the interaction is significant, you need to use judgment to identify the values of the predictor and moderator with reference to which the interaction will have practically significant effects.

E.g., if the efficacy of a medical intervention is conditional on (interacts with) the age of the patient, then you should estimate (with appropriate measures of precision, as well, likely signified by confidence intervals) how big the likely difference in efficacy is likely to be within age ranges in which a practitioner might be contemplating the treatment. Depending on how big that effect is within the relevant age range, one might or might not view the interaction as being a factor that effects the treatment decision. As Peter Flom notes, you are likely to be able to illustrate something like that most effectively with a graph; indeed, reporting an interaction only in a table is also a symptom of mechanized & ritualistic analysis, since the outputs in the table are unlikely to make the meaning and practical significance of an interaction clear.

Rather than identify some text that recommends something that seems dopey, I prefer to recommend a text that sets out an approach that strikes me as sensible. Both

Jaccard, J. & Turrisi, R., Interaction Effects in Multiple Regression, (Sage Publications, Thousand Oaks, Calif., 2003); and

Aiken, L.S., West, S.G. & Reno, R.R. Multiple Regression: Testing and Interpreting Interactions, (Sage Publications, Newbury Park, Calif., 1991)

are excellent. They note the "region of significance" technique but both treat it as pretty unimportant relative to using judgment to probe and explicate interactions once one has identified that they exist.

$\endgroup$
  • $\begingroup$ Now that makes a lot of sense, doesn't it? I agree with your observation about "computational ritual" displacing "judgement". This is my view too but there seems to be an obsession with statistical testing to prove everything. I like commonsense! $\endgroup$ – Adhesh Josh Nov 24 '11 at 20:10
  • $\begingroup$ Adhesh, I am pleased to see you are learning from the questions you have asked here. The initial set (several dozen of them, as I recall) appeared to focus entirely on the computational ritual of hypothesis tests to compare correlation coefficients. You might enjoy reviewing those early questions and the reactions to them, to see how far you have come. $\endgroup$ – whuber Nov 25 '11 at 15:58
1
$\begingroup$

I don't know what you mean by "real or not" - it would help if you quoted the text.

Graphics are often a very good way to understand interactions. They can be hard to understand from just the numbers.

$\endgroup$
  • $\begingroup$ Perhaps I should have said that it did not occur by chance. $\endgroup$ – Adhesh Josh Nov 24 '11 at 20:05
  • $\begingroup$ @AdheshJosh If you know that it did not occur by chance, then you do not need to get any p-values - in fact, all p-values are then 0, because the p-value is only about the case where the null hypothesis is true. $\endgroup$ – Peter Flom Nov 25 '11 at 22:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.