I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ensures $FDR\leq \alpha$ necessarily reject any hypothesis rejected by a procedure that ensures $FWE\leq \alpha$? The Benjamini and Hochberg (1995) method rejects at least as much as Holm (1979) and Bonferroni methods, but it is unclear to me if this would be true for arbitrary methods controlling FDR and FWE.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ "weakly less than"? $\endgroup$– AlexisDec 13, 2021 at 2:28
-
1$\begingroup$ By which I mean $\leq$ $\endgroup$– Golden_RatioDec 13, 2021 at 4:57
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
7
Ok I found that the Benjamini–Yekutieli (2001) procedure, which controls FDR under arbitrary dependence of p values, uses a lower cutoff than Benjamini and Hochberg (1995) and apparently lower than Holm, implying that some nulls rejected by Holm may get accepted by Benjamini–Yekutieli (2001).
-
1$\begingroup$ You probably want to read this question and the most up-voted answers, and the comments, before you get too excited about Benjamini-Yekutieli. The OP feels that no one there has come up with an intuitive example where Benjamini-Hochberg would not apply, but Benjamini-Yekutieli would. BY seems to me to be reaching for some kind construct which may not actually exist. Note: no accepted answer. $\endgroup$– AlexisDec 13, 2021 at 2:31
-
$\begingroup$ @Alexis Thank you for the comment. But putting aside whether there is plausible setting where BY would apply and BH would not apply, on purely theoretic grounds, would you agree that BY may possibly reject fewer nulls than Holm? $\endgroup$ Feb 10, 2022 at 21:43
-
$\begingroup$ Oh, for sure. BY is quite conservative by comparison. I just do not know when it is appropriate to use it. (Pedantic aside: I never say "accept the null" for reasons. :) $\endgroup$– AlexisFeb 10, 2022 at 21:52
-
$\begingroup$ @Alexis Fair enough. The reason I was interested in this is because in the 1995 Benjamini Hochberg paper, they mention that since any procedure that controls FWE also controls FDR, "a gain in power may be expected" from an FDR controlling procedure. But this remark didn't seem to follow logically since strictly speaking, an FDR controlling procedure may still not reject as much as an FWE controlling procedure. $\endgroup$ Feb 10, 2022 at 22:33
-
$\begingroup$ I do not agree: what FWER applies under the conditions where BY applies? I think there's an apples to oranges there. $\endgroup$– AlexisFeb 10, 2022 at 23:20