Why may results from model with interaction term and stratified model be different?

Question

Suppose I wanted to explore the relationship between smoking (X; yes/no) and an disease outcome (Y; eg. visual analogue scale of depression from 0 to 10). But, I know that irrespective of X, Y is different between men and women.

I then hypothesise that the effect of smoking on depression (coefficient of X) will be different between males and females.

I can test this either by running the regression model for each gender, or by adding a gender-by-X interaction term (Y=b0 + b1*X + b2*gender + b3*(X*gender))

Conceptually they are the same, except I can compare regression coefficients in the latter (significance of the difference signified by b3) and only eye-ball the former.

However, the coefficients from these two models are (similar but) not the same! Is there an error in my intuition?

To complicate things further, I have previously seen comparisons of coefficients for each gender being tested using Y=b0 + b1*X + b2*(X*gender). Would I be correct in saying that this method is fine when the intercepts are the same for gender stratified models, but not if they are not (as is in my example)?

Answer 1

In general, the stratified model requires more power to estimate, is more flexible and general, but harder to draw inference from. You cannot directly calculate a $p$-value. You must either use a path-model, bootstrap or permutation test, or the $\delta$-method to obtain standard errors for the difference in regression parameters between two stratified models. A test which is inefficient but often performed is inspecting the coverage of 95% CIs. If either CI overlaps the point estimate of the other stratum coefficient, do not reject the null. This does not provide a powerful test. Inspecting the narrower CI only makes an anticonservative test.

The interaction model by contrast is more efficient, requires homoscedasticity between the two strata, and the inference is just based on the product-term coefficient. The interaction model has the additional advantage that it can assess interaction between two continuous covariates. You couldn't, for instance, fit a stratified model for a continuous covariate without splitting one of the covariates into an arbitrary number of strata; this likely spends too much power. The homoscedasticity assumption (both between strata and overall) can be relaxed by using robust sandwich variance estimation.

These models will consistently estimate the same thing only when the mean model is true. If, in fact, the mean model is not true, it is possible that, as the sample size grows to infinity, one model definitively says there's no interaction and the other says there is interaction. For that reason, with either model, you should perform some diagnostic comparisons to ensure there aren't egregious departures from the modeled mean.

If there are further adjustment variables: the need for correct model specification in the interaction model and power drop in the stratified models is exacerbated. In a simple $X,W,Y$ analysis, the stratified models in total have 4 parameters and the interaction model has 4 parameters (intercept, X coefficient, W coefficient, and product-term: you must always adjust for the main effects). However, introduce adjustment variables $Z_1, \ldots, Z_p$, and the interaction model has 4+p parameters and the stratified model has 2+2*p parameters.

Answer 2

I delved quite into this issue and ended up putting out a paper on it, which may be helpful to you.

https://ehp.niehs.nih.gov/doi/10.1289/EHP334

Basically, the stratified and product-term models encode different assumptions about the covariates. If you were to include product terms between the modifier and all covariates, that is functionally equivalent to a stratified model.