4
$\begingroup$

I've been trying to clarify for myself how to interpret the p values produced by p.adjust with method="fdr/BH".

I'm aware of this question: https://stackoverflow.com/questions/10323817/r-unexpected-results-from-p-adjust-fdr and Multiple hypothesis testing with FDR in R - FDRtool and p.adjust and other similar discussions online, e.g. https://support.bioconductor.org/p/49864/

However, from this it's not clear how to interpret the values produced. It seems like the values this method produces are not really probabilities anymore - as are the case with other 'adjusted' p values, e.g. via Holm method. Rather, it's that if choose to reject null hypotheses where the p_fdr < X, then we are maintaining a false discovery rate < X. Is this the correct interpretation?

$\endgroup$
2
  • $\begingroup$ Could you help us understand the implicit distinction you make between a (presumably long term, hypothetical) rate, as in "false discovery rate," and a probability? $\endgroup$ – whuber Dec 12 '17 at 15:12
  • $\begingroup$ Hmmm. I suppose I'm getting confused because this en.wikipedia.org/wiki/False_discovery_rate#Benjamini–Hochberg_procedure implies the BH procedure controls the familywise FDR rate provided you reject all tests which don't meet the criterion. But I wasn't clear that the values produced then related to a probability for the specific null hypothesis. $\endgroup$ – bjw Dec 13 '17 at 16:40
2
$\begingroup$

By my understanding, the question is about what these corrected p-values actually mean. It would be nice to have, like in a case of testing a single hypothesis, a straight forward meaning such as the probability of observing a given outcome assuming that the null hypothesis is true.

I believe that you are correct in your assertions. In case of the BH correction and other corrections based on controlling the False Discovery Rate , correcting the p-values is just a construct to control the FDR. The same way other methods based on FWER control only serve that purpose and p-values loose their original meaning.

I would argue that the process of making and reporting the correction is here out of habit/ for consistency. It is simple for people to interpret a test result based on a p-value and this way they can keep the way of evaluating statistical significance and make a good judgment.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.