I've been trying to clarify for myself how to interpret the p values produced by p.adjust with method="fdr/BH".

I'm aware of this question: https://stackoverflow.com/questions/10323817/r-unexpected-results-from-p-adjust-fdr and Multiple hypothesis testing with FDR in R - FDRtool and p.adjust and other similar discussions online, e.g. https://support.bioconductor.org/p/49864/

However, from this it's not clear how to interpret the values produced. It seems like the values this method produces are not really probabilities anymore - as are the case with other 'adjusted' p values, e.g. via Holm method. Rather, it's that if choose to reject null hypotheses where the p_fdr < X, then we are maintaining a false discovery rate < X. Is this the correct interpretation?

  • $\begingroup$ Could you help us understand the implicit distinction you make between a (presumably long term, hypothetical) rate, as in "false discovery rate," and a probability? $\endgroup$
    – whuber
    Commented Dec 12, 2017 at 15:12
  • 1
    $\begingroup$ Hmmm. I suppose I'm getting confused because this en.wikipedia.org/wiki/False_discovery_rate#Benjamini–Hochberg_procedure implies the BH procedure controls the familywise FDR rate provided you reject all tests which don't meet the criterion. But I wasn't clear that the values produced then related to a probability for the specific null hypothesis. $\endgroup$
    – bjw
    Commented Dec 13, 2017 at 16:40

1 Answer 1


By my understanding, the question is about what these corrected p-values actually mean. It would be nice to have, like in a case of testing a single hypothesis, a straight forward meaning such as the probability of observing a given outcome assuming that the null hypothesis is true.

I believe that you are correct in your assertions. In case of the BH correction and other corrections based on controlling the False Discovery Rate , correcting the p-values is just a construct to control the FDR. The same way other methods based on FWER control only serve that purpose and p-values loose their original meaning.

I would argue that the process of making and reporting the correction is here out of habit/ for consistency. It is simple for people to interpret a test result based on a p-value and this way they can keep the way of evaluating statistical significance and make a good judgment.

  • $\begingroup$ I think I agree with this answer until the last sentence; I'm not convinced that interpreting the adjusted p-values, after they lose their quantitative meaning outside of maintaining an FDR/expected FDR, is simple or conducive to good judgement. $\endgroup$ Commented Jan 2 at 16:39
  • $\begingroup$ It took me a long time to realize a major potential advantage of adjusting p to get it so that an alpha-level test against the adjusted p maintains an FDR or FWER... it is so that readers can each apply their own threshold to a single set of results. I'm not sure this outweighs the negatives of adjusting p-values. $\endgroup$ Commented Mar 21 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.