I am trying to calculate the type 1 error of a bootstrap hypothesis test procedure (won't go into the hypothesis test here).
I ran the test on 1000 simulated datasets (simulated under the null hypothesis of no effect). The smallest p value I got was 0.45, giving my procedure a type 1 error rate of <0.001.
Is this a contradiction? Shouldn't I find p<0.05 in about 5% of the datasets?