Resources for building explanatory regression models?

Question

Does anyone have any good resources for building explanatory regression models? Using the distinction between explanatory vs. predictive models described in Shmueli (2010) (text available here), I'm wanting to building an explanatory model because I'm interested in examining how several variables specially affect the outcome variable. I'm asking this question here because although I've found advice for building predictive models (e.g., by Gelman and Hill (2006): http://www.stat.columbia.edu/~gelman/arm/), and I'm thinking this process might be different for explanatory models.

To put this question in more context, I have four variables of interest. Plus, I'm expecting these variables to have different effects for males and females, and I'd like to also test some additional covariates that may or may not be important. What I'd like is a guide on how to best build models with these variables.

For example, in an ANOVA approach, I think the convention is to just add the 5 variables (4 I care about and gender) all at once and test the highest order interaction possible (i.e., fit a complex model right off the bat). Does this make sense to apply to a multiple regression framework? If so, when would I add the covariates? These are the types of questions I'm trying to resolve.

Thanks!

---

Gelman, A. and Hill, J., 2006. Data analysis using regression and multilevel/hierarchical models. Cambridge university press.

Shmueli, G., 2010. To explain or to predict?. Statistical science, 25(3), pp.289-310, doi:10.1214/10-STS330.

Answer 1

One major difference between prediction and explanation is that it's hard to interpret interactions of order higher than two, even if they may be important for a better prediction. In addition, you may wish to restrict the model pool to hierarchical models only (if A*B is included, then both main effects, A and B, should be included).

Other than that, you can use any method you want (stepwise, forward selection, backward elimination, AIC, etc). I find that combining backward elimination with AIC works fine: you drop the term with the largest p-value until you find the model with the lowest AIC. To interpret AIC values correctly, look up how Akaike weights are defined and calculate them for two or more competing models.

Another possibility is to use CART which is also interpretable if you limit yourself to a single tree. That way, you can essentially consider higher order interactions without turning it into a black box.