# (LOGIT) Interaction term between Treatment and Female is not significant, but the treatment fx differs when running separate regressions by gender

I am simulating a DGP like this:

$$\operatorname{logit}[p(y) = 1] = \beta_0 + \beta_1\textrm{Treatment} + \beta_2\textrm{Treatment}\times\textrm{IsFemale} + \beta_3\textrm{ IsFemale} + \beta_4 \textrm{IsSmoker}$$ and assume I know there is an interaction effect.

library('paramtest')
library('pwr')
library('ggplot2')
library('knitr')
library('nlme')
library('lavaan')
library('dplyr')
library('lmtest')

set.seed(416)

N= 500

p_to_log_odds <- function(p){log(p / (1-p))}
b.0 <-p_to_log_odds(0.03)
b.treatment <- log(1.5)
b.is_smoker <- log(1.3)
b.is_female <- log(1.3)
b.is_female_treated <- log(1.8)

X  <- data.frame(
treatment = c(sample(c(0,1), N, replace=TRUE)),
is_smoker = c(sample(c(0,1), N, replace=TRUE)),
is_female = c(sample(c(0,1), N, replace=TRUE))
)

X$$lin_pred <- b.treatment * X$$treatment +
b.is_smoker * X$$is_smoker + b.is_female * X$$is_female +
b.is_female_treated * X$$is_female * X$$treatment

X$$pr <-1/(1+exp(-X$$lin_pred))
X$$y = rbinom(N,1,X$$pr)

# MODELS
###########
# Interaction
model.interaction <- glm(y ~ treatment + treatment*is_female + is_smoker + is_female, data=X)
print(summary(model.interaction))


When I simulate this DGP, I get a non significant interaction term.

Call:
glm(formula = y ~ treatment + treatment * is_female + is_smoker +
is_female, data = X)

Deviance Residuals:
Min       1Q   Median       3Q      Max
-0.8741  -0.5327   0.2073   0.3934   0.5058

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)          0.49421    0.04918  10.048   <2e-16 ***
treatment            0.11243    0.05892   1.908   0.0569 .
is_female            0.03848    0.06055   0.636   0.5254
is_smoker            0.08136    0.04153   1.959   0.0507 .
treatment:is_female  0.14760    0.08337   1.771   0.0773 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


But if I run models just for males and just for females I see the treatment is only significant for females.

# Subset just females
model.fem_subset <- glm(y ~ treatment + is_smoker, data=X %>% filter(is_female==1))
print(summary(model.fem_subset))

# Subset just males
model.male_subset <- glm(y ~ treatment + is_smoker, data=X %>% filter(is_female==0))
print(summary(model.male_subset))

Call:
glm(formula = y ~ treatment + is_smoker, data = X %>% filter(is_female ==
1))

Deviance Residuals:
Min       1Q   Median       3Q      Max
-0.8842  -0.5223   0.1158   0.3746   0.4777

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  0.52234    0.04715  11.079  < 2e-16 ***
treatment    0.25880    0.05568   4.648 5.47e-06 ***
is_smoker    0.10308    0.05567   1.852   0.0653 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for gaussian family taken to be 0.1915177)

Null deviance: 51.919  on 247  degrees of freedom
Residual deviance: 46.922  on 245  degrees of freedom
AIC: 298.89

Number of Fisher Scoring iterations: 2

Call:
glm(formula = y ~ treatment + is_smoker, data = X %>% filter(is_female ==
0))

Deviance Residuals:
Min       1Q   Median       3Q      Max
-0.6775  -0.5653   0.3225   0.3826   0.4947

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  0.50526    0.05650   8.942   <2e-16 ***
treatment    0.11219    0.06210   1.807    0.072 .
is_smoker    0.06003    0.06162   0.974    0.331
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for gaussian family taken to be 0.2390443)

Null deviance: 60.520  on 251  degrees of freedom
Residual deviance: 59.522  on 249  degrees of freedom
AIC: 359.49

Number of Fisher Scoring iterations: 2



So if this is the DGP, is it generally better to just conduct two separate regressions instead of estimating an interaction?