Insignificant interaction term - but only for low values of main effect - decline hypothesis?

Question

I run a regression with multiple predictors. I am interested in relationship satisfaction (fictional example). I use gender and hours of time spent together (continuous) as predictors. In my hypothesis I assume that time spent together will have a positive effect on relationship satisfaction, but I assume that this effect differs by gender. So I add an interaction effect. Both main effects are significant and the interaction effect is not. I'm fine with that and normally I would decline my hypothesis. But through margins plots I saw that the insignificance only refers to a few lower values of my continuous variable. Higher values show significant differences by gender. What does that mean for my hypothesis testing? Is it just partially confirmed/declined?

Results are:

           Hazard Ratio   rob.SD   z-Score       p
gender          0.85       0.06     -2.36      0.018
time            1.19       0.02      9.08      0.000
gender*time     0.92       0.05     -1.45      0.147

Answer 1

This is not so surprising. Because you specify different slopes per gender, you will find big differences when you extrapolate. Here is an example

library(tidyverse)

set.seed(0)
d = tibble(
  t = rnorm(25, 0, 1),
  g = rbinom(25, 1, 0.5)
)

d$y = 2*d$t + d$g + rnorm(25, 0, 2)

model = lm(y ~ t*g, data = d)

modelr::data_grid(d, t, g) %>% 
  modelr::add_predictions(model) %>% 
  ggplot(aes(t, pred, color = factor(g)))+
  geom_line()

The true process has no interaction, but allowing the model to have an interaction results in large differences when extrapolating despite the failure to reject the null. Small differences taken in big quantities lead to big changes.

This is not to say an interaction does not exist. It may, and as POC has noted you may be under powered to detect it. However, large differences in predictions are completely consistent with the null being true.

Answer 2

It is probably a lack of statistical power. A clue is the $p=.147$ not so far from the significance threshold.

You have to reject the hypothesis of an interaction. Your results suggest that there may be an interaction, but the effect size was smaller than expected and needed more power.