0
$\begingroup$

I'm struggling with the interpretation of a regression model. My advisor wants me to check whether the effect of a treatment dummy varies with several dummy variables in one model (female dummy, high education dummy and high income dummy).

The regression model looks like this:

$$ y = b_0 +b_1 treatment+b_2 female+b_3 higheduc+ b_4 highincome+b_5 treatment \cdot female +b_6 treatment \cdot higheduc +b_7 treatment \cdot highincome + errorterm $$

I know that in a model with only one interaction (e.g. of female and treatment), the interaction term shows me how much the treatment effect differs among females relative to the basecategory of males. But how is it when I have more than one interaction of the treatment dummy with demographics? Is this still interpretable as the average difference in the treatment effect between gender groups?

I do not think so because the treatment dummy should then show the treatment effect for all interaction dummies equal to zero, i.e. for males having no high income and no high education?

In my view I would have to run 3 regressions, one for each check whether the treatment effect varies with the demographics. But maybe you can correct me :)

Best regards!

$\endgroup$

1 Answer 1

1
$\begingroup$

It's almost never a good idea to run multiple separate sub-models on the same data. You run a risk of omitted-variable bias in a sub-model that leaves out an outcome-associated predictor. You are implicitly averaging over the cases that should be distinguished based on the values of the omitted predictors, but aren't because you left those predictors out. If the distribution of those cases in your data set isn't representative of the underlying population of interest, you can get into even more trouble.

The most efficient use of your data is to build a single model with all predictors and interactions of interest, provided that you have enough data to avoid overfitting. You then can get the most reliable predictions for any specified set of predictor values, and proceed accordingly to generalize to the underlying population.

As you note, there is a problem is in interpreting interaction coefficients, for example $b_5$ for the female by treatment interaction, in this situation. It is "how much the treatment effect differs among females relative to the base category of males" when all other predictors are held constant. In your model there are no other interactions involving female, so that's the same treatment-effect difference for female within any class of high education or high income.

That coefficient, however, is not necessarily an "average difference in the treatment effect between gender groups." Say that there is a significant $b_6$ coefficient for the high education by treatment interaction, and that female cases are relatively over-represented in the high education class. Then the "average treatment effect between gender groups" has to take into account the interaction coefficients and gender imbalances associated with all the predictors interacting with treatment.

The solution to getting "averages," however, is not to use separate sub-models with individual interaction terms, for the reasons outlined in the first paragraph. Get a full model, think carefully about what types of "averages" you want to examine, and evaluate the reliability of those "average" values by taking into account the (co)variance matrix of the coefficient estimates.

$\endgroup$
4
  • $\begingroup$ Hi EdM! Thank you very much for your very elaborate reply! :) Am I understanding correctly that the interaction of female x treatment, in the presence of the other interactions (in the "full model") with the treatment dummy, is then the way to go when I want to estimate the ceteris paribus effects of being female, highly educated, and rich on the treatment effect? It is, as you pointed out, a bit different from doing the seperate models. But I'm not completely understanding what gives me the more "reliable" estimate (just of gender) on the female x treatment. $\endgroup$
    – Nepolo
    Commented May 19, 2021 at 8:30
  • $\begingroup$ @Nepomuk what type of "reliable estimate" of the gender x treatment effect do you want? Ceteris paribus, in your model the $\beta_5$ interaction coefficient is the estimate. That's a "conditional" estimate, given a set of values of the other predictors. It's not the same as a "marginal" estimate: the average difference in treatment effect between males and females across the population. Different treatment effects associated with education or income and male/female differences in education or income determine the "average" difference over the population. Which estimate do you want? $\endgroup$
    – EdM
    Commented May 19, 2021 at 11:47
  • $\begingroup$ @Nepomuk for an introduction to "marginal" versus "conditional" estimates, see for example this page. Do you want an estimate of the average over the population, combining the influences of all the other factors associated with gender? Or do you want to get the estimate in any situation where all of those other factors are held constant? My sense is that you are more interested in the latter, but that should be a conscious choice on your part. $\endgroup$
    – EdM
    Commented May 19, 2021 at 11:54
  • $\begingroup$ Thank you very very much! I think I have to go the conditional way because of too few data :) $\endgroup$
    – Nepolo
    Commented Jun 1, 2021 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.