I am performing a large number of likelihood ratio tests for whether a value of a parameter (let's call it $p$) in my statistical model is smaller or greater than a particular value (1). The underlying distribution is such that in most cases $p < 1$ and these cases can be identified with more confidence (p-values from a LRT in those cases are generally much lower). However, I'm mostly interested in the cases where $p > 1$ and would like to be able to identify as many of them as possible with a reasonable accuracy.

Because of this, I'm unsure how to perform multiple testing correction (I'm using Benjamini-Hochberg). I can think of the following ways of doing this:

1) Perform a single multiple testing correction for all tests 2) Divide the tests into two separate populations, those where the point estimate of $p$ is less than 1 and those where $p > 1$ and perform multiple testing corrections on the two subsets separately 3) Randomise the p-values on sites where the point estimate of $p$ is less than 1 and then perform a multiple testing correction on the entire dataset.

I'm pretty certain that 3) is correct but it is also very conservative. Can 1), 2) or perhaps another approach altogether be justified?

  • $\begingroup$ Maybe the LRT isn't the most appropriate test; can you use a 1-sided test for p>1? $\endgroup$ – qdjm Mar 11 '16 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.