# Why may results from model with interaction term and stratified model be different?

Suppose I wanted to explore the relationship between smoking (X; yes/no) and an disease outcome (Y; eg. visual analogue scale of depression from 0 to 10). But, I know that irrespective of X, Y is different between men and women.

I then hypothesise that the effect of smoking on depression (coefficient of X) will be different between males and females.

I can test this either by running the regression model for each gender, or by adding a gender-by-X interaction term (Y=b0 + b1*X + b2*gender + b3*(X*gender))

Conceptually they are the same, except I can compare regression coefficients in the latter (significance of the difference signified by b3) and only eye-ball the former.

However, the coefficients from these two models are (similar but) not the same! Is there an error in my intuition?

To complicate things further, I have previously seen comparisons of coefficients for each gender being tested using Y=b0 + b1*X + b2*(X*gender). Would I be correct in saying that this method is fine when the intercepts are the same for gender stratified models, but not if they are not (as is in my example)?

• Yes, both approaches should give you the same coefficients and yes, for the comparison it is better to include the level of gender and not just the interaction. Mar 12, 2018 at 13:59
• @AndreasDzemski they will not give you the same coefficients because the squared errors are weighted in the stratified models differently than in the pooled model with product term. Mar 12, 2018 at 14:16

## 2 Answers

In general, the stratified model requires more power to estimate, is more flexible and general, but harder to draw inference from. You cannot directly calculate a $p$-value. You must either use a path-model, bootstrap or permutation test, or the $\delta$-method to obtain standard errors for the difference in regression parameters between two stratified models. A test which is inefficient but often performed is inspecting the coverage of 95% CIs. If either CI overlaps the point estimate of the other stratum coefficient, do not reject the null. This does not provide a powerful test. Inspecting the narrower CI only makes an anticonservative test.

The interaction model by contrast is more efficient, requires homoscedasticity between the two strata, and the inference is just based on the product-term coefficient. The interaction model has the additional advantage that it can assess interaction between two continuous covariates. You couldn't, for instance, fit a stratified model for a continuous covariate without splitting one of the covariates into an arbitrary number of strata; this likely spends too much power. The homoscedasticity assumption (both between strata and overall) can be relaxed by using robust sandwich variance estimation.

These models will consistently estimate the same thing only when the mean model is true. If, in fact, the mean model is not true, it is possible that, as the sample size grows to infinity, one model definitively says there's no interaction and the other says there is interaction. For that reason, with either model, you should perform some diagnostic comparisons to ensure there aren't egregious departures from the modeled mean.

If there are further adjustment variables: the need for correct model specification in the interaction model and power drop in the stratified models is exacerbated. In a simple $X,W,Y$ analysis, the stratified models in total have 4 parameters and the interaction model has 4 parameters (intercept, X coefficient, W coefficient, and product-term: you must always adjust for the main effects). However, introduce adjustment variables $Z_1, \ldots, Z_p$, and the interaction model has 4+p parameters and the stratified model has 2+2*p parameters.

• thank you very much. I've had reviewers suggest correcting for multiple testing in the past. Because interpreting coefficients of interaction models is like doing separate stratified models (my example was for gender). I can see his/her point but feel the suggestion is incorrect. Would you dis/agree? Mar 28, 2018 at 8:13

I delved quite into this issue and ended up putting out a paper on it, which may be helpful to you.

https://ehp.niehs.nih.gov/doi/10.1289/EHP334

Basically, the stratified and product-term models encode different assumptions about the covariates. If you were to include product terms between the modifier and all covariates, that is functionally equivalent to a stratified model.

• Impressive! Many thanks Oct 12, 2019 at 18:25