# What are best practices in identifying interaction effects?

Other than literally testing each possible combination of variable(s) in a model (x1:x2 or x1*x2 ... xn-1 * xn). How do you identify if an interaction SHOULD or COULD exist between your independent (hopefully) variables?

What are best practices in attempting to identify interactions? Is there a graphical technique that you could or do use?

• You could tell us a little bit about your data? size (cf my answer) and nature (cf Gavin's answer) – robin girard Nov 25 '10 at 10:09
• @Robin: Give him time to get out of bed, Brandon is in Toronto ;-) – Reinstate Monica - G. Simpson Nov 25 '10 at 11:41
• @Robin, I would prefer to keep it more general. If, in your response you're providing a method that requires an assumption about the size or nature of data, please state it. The problem I'm having spans a number of different modelling tasks, all with different data. So, in this case, I'm looking for general recommendation on identifying interaction effects. – Brandon Bertelsen Nov 25 '10 at 15:24

Cox and Wermuth (1996) or Cox (1984) discussed some methods for detecting interactions. The problem is usually how general the interaction terms should be. Basically, we (a) fit (and test) all second-order interaction terms, one at a time, and (b) plot their corresponding p-values (i.e., the No. terms as a function of $1-p$). The idea is then to look if a certain number of interaction terms should be retained: Under the assumption that all interaction terms are null the distribution of the p-values should be uniform (or equivalently, the points on the scatterplot should be roughly distributed along a line passing through the origin).

Now, as @Gavin said, fitting many (if not all) interactions might lead to overfitting, but it is also useless in a certain sense (some high-order interaction terms often have no sense at all). However, this has to do with interpretation, not detection of interactions, and a good review was already provided by Cox in Interpretation of interaction: A review (The Annals of Applied Statistics 2007, 1(2), 371–385)--it includes references cited above. Other lines of research worth to look at are study of epistatic effects in genetic studies, in particular methods based on graphical models (e.g., An efficient method for identifying statistical interactors in gene association networks).

### References

• Cox, DR and Wermuth, N (1996). Multivariate Dependencies: Models, Analysis and Interpretation. Chapman and Hall/CRC.
• Cox, DR (1984). Interaction. International Statistical Review, 52, 1–31.

My best practice would be to think about the problem to hand before fitting the model. What is a plausible model given the phenomenon you are studying? Fitting all possible combinations of variables and interactions sounds like data dredging to me.

• sounds like a remark or is the answer "think" ? – robin girard Nov 25 '10 at 8:15
• @Robin - the latter. I find statistical modelling quite difficult (I'm an ecologist with little formal statistical training, most of what I've learned has been self-taught) but it is a lot easier if I think about the problem first, determine what is plausible, build that model, do my model diagnostics, try interactions where these make scientific sense. – Reinstate Monica - G. Simpson Nov 25 '10 at 8:25
• @Brandon: If there is a missing interaction, there will be patterns in the residuals conditional upon values of the covariates. Plotting residuals against the covariates may help determine where an interaction might be appropriate. – Reinstate Monica - G. Simpson Nov 25 '10 at 17:41
• @Brandon: This is standard model diagnostics and exploratory plotting skills. I would plot the residuals against one of the covariates I think might be a candidate for an interation, conditioned (in the ggplot2 or lattice way) on the values of the covariate I think is involved in the interaction. Stick a loess smoother through each panel to see if there are patterns. Depends on what type of variables your covariates are. – Reinstate Monica - G. Simpson Nov 27 '10 at 9:20
• Data dredging? If you torture the data long enough, it will confess... – Curious Jan 18 '13 at 10:57

Fitting a tree model (i.e. using R), will help you identify complex interactions between the explanatory variables. Read the example on page 30 here.

• Very simple and very useful. Thanks for the reference to Crawley's text as well! – Brandon Bertelsen Nov 25 '10 at 16:21
• Be careful - you can't easily fit those kinds of interactions in say a linear model. The interactions occur only in one branch of the tree (or part of). You need a lot of data to use these sorts of tools in real world data. – Reinstate Monica - G. Simpson Nov 25 '10 at 17:38
• As @Gavin said, one of the potential pitfalls is that decision trees need a large sample size and are quite unstable (which is one of the reason bagging and random forests were proposed as viable alternatives). Another problem is that it is not clear whether we seek for second- or higher-order interaction effects. In the former case, CARTs are not a solution. In any case, I will find very doubtful any interpretation of an interaction between 6 variables in any kind of study (observational or controlled). – chl Nov 26 '10 at 20:43

I'll preface this response as I entirely agree with Gavin, and if you're interested in fitting any type of model it should be reflective of the phenomenon under study. What the problem is with the logic of identifying any and all effects (and what Gavin refers to when he says data dredging) is that you could fit an infinite number of interactions, or quadratic terms for variables, or transformations to your data, and you would inevitably find "significant" effects for some variation of your data.

As chl states, these higher order interaction effects don't really have any interpretation, and frequently even the lower order interactions don't make any sense. If your interested in developing a causal model you should only include terms you believe could be pertinent to your dependent variable A priori to fitting your model.

If you believe they can increase predictive power of your model, you should look up resources on model selection techniques to prevent over-fitting your model.

How large is $n$ ? how many observations do you have ? this is crucial ...

Sobol indices will tell you the proportion of variance explained by interaction if you have a lot of observations and a few $n$, otherwise you will have to do modelling (linear to start with). You have a nice R package for that called sensitivity. Anyway the idea is quite often that of decomposing the variance (also called generalized ANOVA).

If you want to know if this proportion of variance is significant you will have to do modelling (roughly, you need to know the number of degrees of freedom of your model to compare it to the variance).

Are your variables discrete or continuous ? bounded or not really (i.e you don't know the maximum) ?

• thanks for the direction to Sobol indices. Again, I'd like to specify that I'm looking for a general rather than a specific answer here. I'm not asking about a specific set of data but rather trying to explain a problem I've been having with a number of different sets. – Brandon Bertelsen Nov 25 '10 at 15:52