# Joint model with interaction terms vs. separate regressions for a group comparison

After gathering valuable feedback from previous questions and discussions, I have came up with the following question: Suppose that the goal is to detect effect differences across two groups, male vs. female for example. There are two ways to do it:

1. running two separate regressions for the two groups, and employ Wald test to reject (or not) the null hypothesis $H_0$: $b_1-b_2=0$, where $b_1$ is the coefficient of one IV in male regression, and $b_2$ is the coefficient of the same IV in female regression.

2. pool the two groups together, and run a joint model by including a gender dummy and an interaction term (IV*genderdummy). Then, the detection of the group effect will be based on the sign of interaction and the t-test for significance.

What if Ho is rejected in case (1), i.e. group difference is significant, but the coefficient of interaction term in case (2) is statistically insignificant, i.e. group difference is insignifant. Or vice versa, Ho is not rejected in case (1), and interaction term is significant in case (2). I have ended up with this outcome several times, and I was wondering what outcome would be more reliable, and what is the reason behind this contradiction.

Many thanks!

-
the difference between procedures is that one assume same variance for both groups. The separate analysis assumes different variances. –  probabilityislogic Aug 5 '12 at 5:17
Thanks a lot!Are you aware please of any reference discussing the issue of variances when comparing different models? –  Bill718 Aug 5 '12 at 14:03

The first model will fully interact gender with all other covariates in the model. Essentially, the effect of each covariate (b2, b3... bn). In the second model, the effect of gender is only interacted with your IV. So, assuming you have more covariates than just the IV and gender, this may drive somewhat different results.

If you just have the two covariates, there are documented occasions where the difference in maximization between the Wald test and the Likelihood ratio test lead to different answers (see more on the wikipedia).

In my own experience, I try to be guided by theory. If there is a dominant theory that suggests gender would interact with only the IV, but not the other covariates, I would go with the partial interaction.

-
Thanks! Yes, actually there are various covariates, not only one IV, I just mentioned one IV in the question for simplicity. The thing is that there isn't a strong theory that could support interaction between gender and certain covariates, it is exploratory analysis, so I need to experiment with many interactions and model fits; the initial model contains 30 predictors... –  Bill718 Aug 5 '12 at 13:11

Anytime two different procedures are used to test a particular hypothesis there will different p-values. To say one is significant and the other is not can be just making a black and white decision at the 0.05 level. If one test gives a p-value of 0.03 and the other say 0.07 I would not call the results contradictory. If you are going to be that strict in thinking about significance it is easy to have either situation (i) or (ii) arise when boardline significance is the case.

As I mentioned in response to the previous question my preference for looking for an interaction is to do one combined regression.

-
Yes, it is true that the combined regression seems to perform better, at least in my case, and it is a very flexible method, since someone could try with different interactions and model fits.I just wanted, by "statistical" curiosity let's say, to find out what is the reason behind the somehow different results . Regarding p-values, I have heard some people accepting significance only at a=0.5% level or less. I am more flexible, using a=1% level, but the big headache comes when the p-values are completely different. –  Bill718 Aug 5 '12 at 13:39
I have seen studies for example, where one IV is very significant when an ordered logit modet is employed, while the same IV becomes insignificant when an OLS is applied. So, in that case, the explanation of the results can be a bit tricky. Thanks a lot for your comments and feedback! –  Bill718 Aug 5 '12 at 13:40
+1, the point about $0.07\approx 0.03$ is an excellent one. –  gung May 27 at 16:03