3
$\begingroup$

This is, I hope, quite a simple question. Being new to the concepts and methods of Multiple Testing, I've been histogramming some of the p-values I've been computing in my genomics research. What I have been seeing is that the distribution of p-values seems to be depleted at low values. Of course, this is horrid for the various FDR, etc methods.

But, I'm simply curious as to what this implies about my data? Why would I be getting fewer low p-values than expected from the null hypothesis?

The sample size for the test is quite small. I suspect that has something to do with it. These p-values are computed for Cox regression of a clinical variable with a genomic variable.

histogram of p-values for all tests

$\endgroup$
  • 1
    $\begingroup$ It would probably help if you were to describe what procedure and data underly the plot. $\endgroup$ – jona Aug 25 '14 at 18:57
  • 1
    $\begingroup$ They are p-values for Cox regression coefficients. $\endgroup$ – PickledZebra Aug 25 '14 at 19:19
2
$\begingroup$

It looks like underdispersion, which Efron described in a number of papers here:

http://statweb.stanford.edu/~ckirby/brad/papers/

In particular, in this paper:

http://statweb.stanford.edu/~ckirby/brad/papers/2006Size.pdf

in Figure 1 we can see that the right histogram of z-values is too "narrow", which is equivalent to saying that the distribution of p-values is not uniform because the proportion of small p-values is lower than expected.

In that case, you have to adjust for underdispersion by, say, using Efron's package locfdr in R.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.