I have been struggling with the interpretation of the coefficients in my multiple regression model and I hope you can help me with that! Thanks in advance for reading the long explanation that follows and for thinking along!
I have 4 predictors in my model: 1 binary categorical (call it "group", coded as 0/1), 1 categorical with 3 levels, entered as 2 dummies (call it "treatment") and 2 continuous. My model includes all predictors, all 2-way, 3-way and 4-way interactions between them (i.e., product terms for each dummy). I find a significant 4-way interaction. In addition, I find that the 3-way interaction (3-level categorical * continuous * continuous) is significant for one level of the binary variable (group A), but it is not significant for the other (group B). There are also no lower-order effects for group B.
Question 1: Does this justify splitting the file based on the binary variable to separately explore the 3-way interaction in group A (where it is significant, and needs further probing)? Based on my theory, it does, as I predict no effects for group B, and a 3-way interaction for group A, and that's exactly what I find. But is that a statistically sound approach (in this case)?
Question 2: If I use the full model to probe the (4-way) interaction, I need to test the effects of my dummified "treatment" variable at LOW and HIGH values of my two continuous moderators for each of the 2 groups (A and B). To this end, I conduct a spotlight analysis - I center my two moderators at their "low" (-1SD) and "high" (+1SD) values (so that the coefficients in the model could be interpreted as the effect of a predictor when all other variables = 0) and then re-estimate a series of regressions for each combination of group/treatment/moder1/moder2 (every time recoding the categorical variables and plugging in the new high/low-centered moderators).
My hypothesis is that my treatment variable will have different effects at HIGH vs. LOW levels of the moderators. So, I run a model where I have the group variable (0/1), the two dummies (0/1), the HIGH mod1, the HIGH mod2, and all their interactions; and another model where I use the LOW-centered moderators. Which coefficient should I look at to see if my hypothesis is supported? The coefficient that quantifies the simple simple effect of my treatment variable (one of the two dummies) within each of the models, or the coefficient for the simple interaction between my treatment variable and the continuous moderator in each of the two models?
What I find is that in one model (at HIGH mod 1 and HIGH mod 2) the 2-way interaction is significant, but in the other (at LOW mod 1& 2) it is not. The simple simple effect of my treatment is not significant. I am not sure how to interpret this.
I will be happy to provide more information, if necessary. Once again, thanks a lot for your help!