# Controlling FDR when p-values are bimodal

I have a quick question about using the Benjamini-Hochberg procedure to control FDR using the p.adjust() function in R. I have read that in order to properly apply an FDR control, your p-value distribution should look something like it does in "Histogram of pLeft". This is, as I understand it, because this and most other algorithms have as an assumption that your null p-values are uniformly distributed.

However, the p-values that I obtained from our experiment (identifying significantly enriched GO Categories between two tissue types) conform to a distribution like the one seen in "Histogram of pBimodal". This is not surprising to us, but does present a problem in that we want to be able to control the FDR given that we have ~3000 independent tests (even more for a second dataset with a similar distribution).

My primary question is "Is there a way to apply a correction given the p-value distribution seen in 'Histogram of pBimodal'"? I've heard that one can apply Benjamini-Hochberg to a p-value vector with a bimodal distribution, but that it will increase the Type-II error rate compared to what it would be were the data unimodal. I've also tested this using simulated data in R, and it appears to hold generally true when the number of hypotheses tested is held constant. I'm much more concerned with the Type-I error rate than the Type-II error rate for our data, if it is a valid approach. That is, I'd rather exclude some truly enriched GO categories than include a number of non-enriched categories. If not, could anyone recommend a different method to go about this?

• Are you aware of this link: varianceexplained.org/statistics/interpreting-pvalue-histogram ? – vkehayas May 4 '18 at 10:46
• I am, that is how I discovered that this is was an issue. However, their suggestions unfortunately don't apply well to this case. I've discovered that the only risk of applying this procedure to my data as it stands is that π<sub>0</sub> will increase, making the correction more conservative. I'm willing to accept the risk of a few more Type-II errors. I'd rather exclude a couple of false-negatives than include a bunch of false-positives. Edit: the "π<sub>0</sub>" should read "π naught". Not sure why that didn't subscript. I guess it just doesn't in comments? – Adam Rork May 4 '18 at 15:58