Regression with sample split

Question

I have a question with respect to running multiple linear regressions for the entire sample and different subsamples:

I have a dataset that includes a dependent variable y and several explanatory variables x1, ..., xN and z. I am interested particularly in the binary explanatory variable z. If I split the entire dataset according to this variable z, I obtain significantly different values of the mean of y for the subsamples conditioning on z = 0 and z = 1.

Now, I would like to run 3 regressions of y on all of x1, x2, ..., xN, and (potentially) z. The first regression is for the entire sample, the second regression for z = 0, and the third regression for z = 1. Should I include z as regressor in the first regression if I would like to compare all 3 regressions or should I omit it to have the same regressors for all three regressions?

Thank you!

Answer 1

It sounds as though you might be exploring a question of moderation. That is to say, ¿does the relationship between a dependent variable and the predictors change for different groups?

As such, the two models you want to explore are the models that predict the dependent variable from all your predictors, $x_1$ ... $x_n$ and your dichotomous grouping variable $z$. Then run the model with the inclusion of the interactions (products) of all your predictors with the dichotomous variable: $z\cdot x_1$ ... $z\cdot x_2$. The significance of the coefficients for this product terms will indicate if there is a different relationship between your two groups or not.

Happy to clarify more if needed.

Answer 2

What is your aim? Are you trying to find variables that will predict Y in the distribution from which your sample is obtained, in particular how good a predictor Z is--or are you try to find the causal influence on Y of predictor variables--especially Z?

if the former, check your distribution assumptions include Z and regress away. Regression is the best linear predictor.

If the latter, then in most settings you should not use multiple regression at all. If there are confounders (unobserved common causes) of the outcome variable and any of your predictors you may get very erroneous results for the confounded variable and others.

That said, if you are going to use regression, I would do it separately conditional on different Z values and also with Z as a predictor. In the last case, regression will tell you if Z is "significant" conditional on all other predictors jointly. Conditional on the two values of Z will tell you what a difference in Z makes for the association of the other predictors with the outcome variables. But if you want to know whether and how much manipulating the value of Z will change Y, regression will only tell you when special conditions are satisfied. For example, for all you know Z might influence Y only via X. When you do the regression including Z and X, the regression coefficient for Z will be 0 (up to sampling variation. Again, Z does not cause Y but Z causes X and X does not influence Y but there is an unmeasured common cause of X and Y, regression will find a "significant" coefficient for Z.

Moral: to use multiple regression for sound causal inference, you have to already know a lot about the causal relations among the variables.