0
$\begingroup$

I have a question about how exactly the Benjamini-Hochberg method for multiple hypothesis testing works. The Wikipedia page for False discovery rate says:

  1. For a given $\alpha$, find the largest k such that $P_{(k)}\leq {\frac {k}{m}}\alpha$
  2. Reject the null hypothesis (i.e., declare discoveries) for all $H_{(i)}$ for $i=1,\ldots ,k$

My question is, what if there's a hypothesis $H_{(i)}$ with $i < k$ but where $P_{(k)} > {\frac {k}{m}}\alpha$ ? Would this hypothesis be called significant? This is probably rare, but for instances where p-values increase with a slope less than $\frac{1}{m}$ (where $m$ is the number of hypotheses tested), it's possible I think.

$\endgroup$
1
  • 1
    $\begingroup$ Yes, this happens and Benjamini-Hochberg then rejects the $i$th null hypothesis. $\endgroup$
    – frank
    Jul 23, 2022 at 9:00

1 Answer 1

2
$\begingroup$

Assuming that you are interested in if it is possible to have $i<k$ and $p_{(i)}>\frac{k}{m}\alpha$, then I would think it is impossible because the Benjamini-Hochberg first reorders all of the p-values from smallest to largest. (I am going to assume all p-values are unique)

$$ p_1,...,p_n\rightarrow p_{(1)},...,p_{(n)} $$

So, if $p_{(k)}\leq \frac{k}{m}\alpha$, this implies that for all $i<k$, $p_{(i)}<p_{(k)}<\frac{k}{m}\alpha$.

$\endgroup$
1
  • $\begingroup$ Right. I understand $p_{(i)} \le p_{(k)} \le \frac{k}{m} \alpha$. just wondering about the case where $p_{(i)} \not \le \frac{i}{m} \alpha$. For example: if we had two hypotheses ($m=2$) with p-values of $0.08, 0.09$, and a FDR $\alpha=0.1$. Then the condition would fail for $i=1$ but succeed for $i=2$. I was wondering what B-H would say in situations like this. $\endgroup$ Jul 26, 2022 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.