1
$\begingroup$

I have the following model:

LOS (length of stay) = b0 + b1 Age + b2 Gender + b3 BMI where Age and Gender are control variables, and my main IV is BMI. And this model isn't made for predictive purpose; we want to know if high BMI causes more LOS.

Results showed insignificant effect of BMI (P > .05).

I then tried to create interaction effects (without any literature-based knowledge): Age x Gender; BMI x Gender; BMI x Age.

Adding these interaction terms results in a significant effect for BMI x Age.

My question is: is it ok to conclude that BMI has no significant effect on LOS, but it does a significant effect on LOS at certain age levels (interaction). I mean is it ok to dig for significant interactions once my effect initially fails to show significance, given no expert / literature knowledge exists about the existence of such interaction effects.

$\endgroup$
2
  • $\begingroup$ May I know why you did not test for the 3 way interaction between BMI, gender and age BMIgenderage ? $\endgroup$
    – CaroZ
    Nov 20, 2023 at 13:41
  • $\begingroup$ What is your sample size ? $\endgroup$ Nov 21, 2023 at 9:52

1 Answer 1

4
$\begingroup$

I mean is it ok to dig for significant interactions once my effect initially fails to show significance, given no expert / literature knowledge exists about the existence of such interaction effects.

In a word, "No".

Digging for statistical significance increases the probability you find at least one statistically significant result, even if there are no true effects to be found.

You should pre-specify all analyses and stick only to those analyses you intend to perform. If you do go ahead with this sort of "digging", I would encourage you to report those analyses exploratory, refrain from reporting p values, and be very modest in conclusions made therefrom.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.