# How to deal with identical p-values with the Benjamini-Hochberg method for correcting for multiple testing

I'm testing for mutations in DNA and I'm using the Benjamini-Hochberg method to modify the threshold I'm testing my p-values against. The method is basically to rank the p-values and compare them to a new threshold defined by

$q(k) = k/n*\alpha$,

where $\alpha$ is your original threshold, commonly 0.05, $k$ is the rank of the p-value you compare with, and $n$ is the total amount of p-values (== total amount of tests).

Due to the nature of my data, many p-values are identical. The following ordered set of p-values could be an example:

$p_1$ = 0.01 $p_2$ = 0.03 $p_3$ = 0.03 $p_4$ = 0.03 $p_5$ = 0.09

Assuming a threshold of $\alpha$ = 0.05, what I have to compare my values to are:

$q_1$ = 0.01 $q_2$ = 0.02 $q_3$ = 0.03 $q_4$ = 0.04 $q_5$ = 0.05

And thus I accept the second test and reject the third and fourth even if they have the same original p-value. I tried to search the literature for a solution to this, but nothing came up. Is there an established method for this? And, if not, what is your preferred ad hoc solution?

-

Verifying also in the multtest package from Bioconductor, I would suggest to give them the same rank - and very importantly - increment the rank by one for the following p-value(s) rather than using their index+1 in an array! This would have the following result:

considering your examplemulttest's BH would rank $r_1$: 1, $r_2$: 2, $r_3$: 2, $r_4$: 2, $r_5$: 3 rather than of $r_2$: 2, $r_3$: 2, $r_4$: 2, $r_5$: 5

-
Thank you. I accept this answer although it is essentially the same as the one by Maciej because of the extra detail. –  Viktiglemma Oct 5 '12 at 12:48
Thank you - I effectively picked up on the comment of Maciej. I often miss the possibility to accept more than one answer! –  dmeu Oct 16 '12 at 13:09

Possible ad hoc solution is to give repeated p-values the same rank.

-
I've just exploring my multtest package results and I see that I have a subset of the same FDR corrected values [1] 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 [10] 0.08199  So maybe it is as I wrote. –  Maciej Jończyk Nov 24 '11 at 19:36