Is the FDR linear-step up procedure Benjamini and Hochberg valid for empircal p-values from non-parametric tests (e.g., bootstrap)?

Question

I have empirical p-values from bootstrapped hypothesis tests with 10,000 resamples each. Because there are a finite number of resamples, some of the p-values are "0", i.e., < 0.0001 with the exact value being unknown (this is the case for 11 out of 30 of my hypothesis tests). Now, my question: is the FDR correction (specifically the Benjamini Hochberg linear step up procedure) valid in this situation where the exact values aren't available for some p-values? In other words, if I have to pretend that my p-values are 0 for those that are < 0.0001, does this invalidate the correction?

Answer 1

Yes, saying your $p$-value is zero is a problem. But you can say it's $1/(B+1)$ (eg, 1/10001).

For some resampling-based tests that's always the right thing to do anyway, and it's not going to do any harm.