Skip to main content
3 of 5
deleted 438 characters in body; edited title
shenflow
  • 1.1k
  • 13
  • 32

Testing multiple hypotheses multiple times

I estimate a linear model consisting of one dependent and four independent variables. I want to test wether each of the independent variables has a significant impact on the dependent variable, i.e. I want to test four hypotheses.

I repeat this test ten times with ten different variations of the dependent variable. How do I adjust p-values in this case?

Do I have to adjust them twice? One time given the fact that I test four hypotheses with one model and one time given the fact that I repeat the test ten times? If the former is true, wouldn't I always have to adjust any p-value in any multiple regression model?

shenflow
  • 1.1k
  • 13
  • 32