Timeline for Multiple Regression: Two Binary ind. vars. - Can an interaction term be significant, when the main effects are not?
Current License: CC BY-SA 4.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 28, 2020 at 7:25 | comment | added | perenniallydisappointed | @StatsStudent You seem to be confused between the existence of main effects in the data generating process and the finding of significant main effects with a model. There are no main effects in my example, because that is how I specified the data generating process. But if I were to model the data it generated with main effects, those main effects could be significant. Indeed, with an alpha level of 5%, they are significant in 5% of random samples - even though they are not there "in the real world". I hope this clarifies things for you. | |
| Aug 27, 2020 at 20:02 | comment | added | StatsStudent | Of course they must be present. In order to have an interaction you have to have the individual effects present in the model -- even your example shows that. They just aren't significant. Frank Harrel puts this nicely here on CV: "not only is it necessary to have all lower order effects in the model when they are connected to higher order effects, but it is also important to properly model main effects that are seemingly unrelated to the factors in the interactions... That's because interactions between x1 and x2 can be stand-ins for main effects of x3 and x4." | |
| Aug 27, 2020 at 10:30 | comment | added | perenniallydisappointed | @ StatsStudent I looked up the hierarchy principle in the source you mention. It does not support your claim that "[main effects] must be present in the real world". I believe the example I gave earlier shows that this claim is false. | |
| Aug 27, 2020 at 1:34 | comment | added | StatsStudent | Also see here for a discussion on this principle: stats.stackexchange.com/questions/27724/… | |
| Aug 27, 2020 at 1:30 | comment | added | StatsStudent | @perenniallydisappointed and Dale, I didn't mean to imply that the main effects are significant. I simply mean that the effects are there. They are present. So it doesn't make sense in my opinion to account for interactions and not have main effects, regardless of how insignificant they might seem, included in the mode, but they must be present in the real world. This is not a new concept either. It's something called the hierarchy principle in design and analysis of experiments. See James, Gareth, Witten, Hastie, and Tibshirani. 2014. An Introduction to Statistical Learning for ex. | |
| Aug 27, 2020 at 0:40 | comment | added | Davis70 | @StatsStudent - I agree that main effects should be reported. I found a general category of solutions that produces this effect, which I posted as a separate answer. | |
| Aug 27, 2020 at 0:38 | comment | added | Davis70 | @whuber Thanks for your answer. I posted a separate answer, which describes situations that produce this effect, rather than, just a small percent of the time. | |
| Aug 24, 2020 at 8:13 | comment | added | perenniallydisappointed | @StatsStudent Though I cannot fault your recommendation for including main effects when looking at interactions, I don't think I's accurate to suggest that main effects must exist when there are interaction effects. For example, imagine a medicine that works for men but not for women. If we code a dummy variable for men as one, then the main effect of treatment is zero and the main effect of the dummy is zero, but the interaction between the two is not. | |
| Aug 23, 2020 at 21:10 | comment | added | StatsStudent | @Dale70, in any event, since you seem to be interested in the interpretation of the effects (or even the interaction), rather than purely predictive modeling, I'd strongly encourage inclusion of any terms that are included in your interactions as this HAS to occur in the real world. | |
| Aug 23, 2020 at 20:59 | comment | added | Davis70 | @StatsStudent I have a rough idea of a different type of implementation, more along the lines of temperature and cooking time affecting cookie quality (a Wikipedia example), but also different - my example would be anti-synergistic at the key points where you would expect significance in the individual main effects - but it's still a vague idea. | |
| Aug 23, 2020 at 20:47 | comment | added | StatsStudent | One thing I think that's worth noting too is that I always include main effects in my models that must be interpreted when interaction effects are found. After all, in the real world, how can an interaction exist and even enter a model if there are no main effects? | |
| Aug 23, 2020 at 19:28 | history | answered | whuber♦ | CC BY-SA 4.0 |