Skip to main content
1 of 5
shenflow
  • 1.1k
  • 13
  • 32

Adjusting for multiple comparions twice

I want to test a set of four hypotheses. I estimate a single model to test all of them (each hypothesis pertains to one coefficient in the model). I repeat this estimation 10 times with 10 slightly different definitions of the dependent variable. I want to reject each null hypothesis if at least one of the 10 repeated tests indicates significance for the respective coefficient.

How do I correctly adjust the p-values in this case? Do I have to adjust them twice? One time based on the fact that I test four hypothesis with one model and one time based on the fact that I repeat the test 10 times?

shenflow
  • 1.1k
  • 13
  • 32