Skip to main content
4 of 5
added 174 characters in body
shenflow
  • 1.1k
  • 13
  • 32

Testing multiple hypotheses multiple times

I estimate a linear model consisting of one dependent and four independent variables. I want to test wether each of the independent variables has a significant impact on the dependent variable, i.e. I want to test four hypotheses. I repeat this test ten times with ten different variations of the dependent variable (the qualitative definition of the dependent variable stays the same). If at least one of ten of the respective p-values indicates significance, I infer that there seems to be a relation between the respective coefficient and the dependent variable.

How do I adjust p-values in this case?

Do I have to adjust them twice? One time given the fact that I test four hypotheses with one model and one time given the fact that I repeat the test ten times? If the former is true, wouldn't I always have to adjust any p-value in any multiple regression model?

shenflow
  • 1.1k
  • 13
  • 32