# Regression with sample split

I have a question with respect to running multiple linear regressions for the entire sample and different subsamples:

I have a dataset that includes a dependent variable y and several explanatory variables x1, ..., xN and z. I am interested particularly in the binary explanatory variable z. If I split the entire dataset according to this variable z, I obtain significantly different values of the mean of y for the subsamples conditioning on z = 0 and z = 1.

Now, I would like to run 3 regressions of y on all of x1, x2, ..., xN, and (potentially) z. The first regression is for the entire sample, the second regression for z = 0, and the third regression for z = 1. Should I include z as regressor in the first regression if I would like to compare all 3 regressions or should I omit it to have the same regressors for all three regressions?

Thank you!

• If z is of interest why would you exclude it? If it is related to your outcome you will get into "omitted-variables bias" territory. May 4, 2023 at 11:44
• That's a good point. And it is not problematic that there is no z in regressions 2 and 3? Or could one also argue that z is implicitly in these regressions as z is the same for all observations in the corresponding subsample, so including z in these regressions does not change the results? May 4, 2023 at 11:47

It sounds as though you might be exploring a question of moderation. That is to say, ¿does the relationship between a dependent variable and the predictors change for different groups?

As such, the two models you want to explore are the models that predict the dependent variable from all your predictors, $$x_1$$ ... $$x_n$$ and your dichotomous grouping variable $$z$$. Then run the model with the inclusion of the interactions (products) of all your predictors with the dichotomous variable: $$z\cdot x_1$$ ... $$z\cdot x_2$$. The significance of the coefficients for this product terms will indicate if there is a different relationship between your two groups or not.

Happy to clarify more if needed.

• Thank you. That's totally correct. I have roughly 30 predictors in the model. Including the interaction effect for each predictor would lead to significantly more coefficients (that I also would need to report, making it even more ugly). From my point of knowledge, the sample split is equivalent to your approach. Correct me if I am wrong. Of course, the interaction terms would directly show if there is a significant difference between the groups. However, I am more interested in which coefficients are really significant for which subsample. Given this, do you think my approach is ok? May 4, 2023 at 14:01
• Others may disagree with me, but I would start with a model-building approach. First, run the two models (with the interactions and without...with all of your variables). Run a $\Delta R^2$ ANOVA to see if the models are different. If you have a significant difference in the models, then look at the interaction terms p-values individually. Drop those that are not significant. Once you have flagged all the potential variables moderated by the grouping variable, check the model again for overall improvement. May 4, 2023 at 14:18

What is your aim? Are you trying to find variables that will predict Y in the distribution from which your sample is obtained, in particular how good a predictor Z is--or are you try to find the causal influence on Y of predictor variables--especially Z?

if the former, check your distribution assumptions include Z and regress away. Regression is the best linear predictor.

If the latter, then in most settings you should not use multiple regression at all. If there are confounders (unobserved common causes) of the outcome variable and any of your predictors you may get very erroneous results for the confounded variable and others.

That said, if you are going to use regression, I would do it separately conditional on different Z values and also with Z as a predictor. In the last case, regression will tell you if Z is "significant" conditional on all other predictors jointly. Conditional on the two values of Z will tell you what a difference in Z makes for the association of the other predictors with the outcome variables. But if you want to know whether and how much manipulating the value of Z will change Y, regression will only tell you when special conditions are satisfied. For example, for all you know Z might influence Y only via X. When you do the regression including Z and X, the regression coefficient for Z will be 0 (up to sampling variation. Again, Z does not cause Y but Z causes X and X does not influence Y but there is an unmeasured common cause of X and Y, regression will find a "significant" coefficient for Z.

Moral: to use multiple regression for sound causal inference, you have to already know a lot about the causal relations among the variables.