I've been using the R package qvalue to estimate q-values based on p-values, and in most cases I have been observing results consistent with what one would expect from multiple testing correction.

However, in one particular instance, qvalue appears to estimate a strangely low pi0est (the estimate of the proportion of true null hypotheses). This vector of p-values is as follows (rounded): 0.014 0.424 0.249 0.706 0.705 0.502 0.359 0.830 0.695 0.447

The pi0est in this instance is 0.033, while the corresponding q-values are (rounded): 0.004 0.026 0.0260 0.026 0.026 0.026 0.026 0.026 0.026 0.027

I am aware that q-values can in some cases be lower than the corresponding p-value, however in this instance I feel that the decrease is far too low to be justifiable. This can be seen in the following plot of q-values (y-axis) vs p-values (x-axis). Further, I don't understand how such a relatively uniform distribution of p-values can lead to such a small pi0 and FDR.

p vs q plot

  • 1
    $\begingroup$ Are you only adjusting on 10 or so p-values? Storey's q-values are data-driven, so I believe these numbers are far too small. I believe I have seen recommendations for over 200 p-values. $\endgroup$ – Moose Oct 26 '16 at 8:46
  • $\begingroup$ I have simulated a larger data set by replicating these values 1,000 times and adding gaussian noise, and I obtained largely similar results (no q-value above 0.1). $\endgroup$ – alan ocallaghan Oct 26 '16 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.