Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

It was suggested to me recently that the significance of an interaction term in a glm has to be higher than a main effect. For example, p<0.05 is commonly thought of as significant for a main effect but a two-way interaction has to be higher (p<0.025 or something) and a three-way even higher. However, I can't find any literature on this so I am unsure. Are they getting confused with multiple comparisons I wonder? Or am I confused and the two things are linked?

share|improve this question
    
I can conceive that someone might choose to apply a more stringent significance level on interactions. Since the number of interactions and higher order interactions grow at a rapid rate (if there are enough factors), I can see some argument for it, such as if one were trying to make a similar number of type I errors at each order. I don't see how that somewhat plausible argument leads to an 'ought', though. –  Glen_b Feb 15 at 23:08

1 Answer 1

up vote 7 down vote accepted

I've never heard that and, to me, it makes little sense.

First there are all the usual problems with p values and cutoffs. But, if anything, I think the p value cutoff for interactions ought to be more lenient than that for main effects, since they can be harder to find if either or both main effects are measured less than perfectly reliably.

share|improve this answer
    
Thanks, I thought so, but it was suggested by a superior so I thought it would be good to triple check things. –  user29689 Feb 15 at 16:57
    
If anything, I've also seen interactions tested at a more lenient significance level (usually p<0.1 in biomedical research). @Peter, do you have a reference for why interactions should be tested more leniently, or is it simply common practice? –  Moose Feb 15 at 17:02
    
I haven't got a reference and I'm not sure how common it is, but it seems to make sense. –  Peter Flom Feb 15 at 17:30
    
Thanks again, but I've just thought of something else.... If the interaction is between categorical variables with multiple levels and I am using contrasts (essentially multiple comparisons) to determine the significance of each level - should I be careful of the p values in that situation? –  user29689 Feb 18 at 9:07
    
I am not sure why that situation would be different from others with multiple comparisons –  Peter Flom Feb 18 at 11:23

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.