5
$\begingroup$

I want to calculate confidence intervals of positive likelihood ratios of multiple dependent tests.

To adjust the confidence level for the 'problem of multiple comparisons', I believe I should control the False Coverage Rate for the confidence intervals, using the Benjamini–Hochberg procedure (or something equivalent) to take into account the positive dependence of the tests.

However, it is not entirely clear to me how to do this for confidence intervals of positive (or negative) likelihood ratios, since the Benjamini–Hochberg procedure requires p-values. Is there a way to convert positive likelihood ratios to p-values or should I do something else?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The idea behind False-Coverage-rate (FCR) adjusted confidence intervals (CIs) is that if you can construct regular CIs, you merely need to inflate them for FCR control. The inflation is done by constructing the marginal CIs to the level $\alpha R/m$ where $\alpha$ is the desired FCR level, $R$ is the number of selected parameters, and $m$ is the number of candidate parameters.

If you are constructing intervals on all considered parameters (i.e., $R=m$), there is no need for correction. Regular intervals will control the FCR [1].

For an application, and brief intro to other FCR controlling methods, see [2].

[1] Benjamini, Y., and D. Yekutieli. “False Discovery Rate-Adjusted Multiple Confidence Intervals for Selected Parameters.” Journal of the American Statistical Association 100, no. 469 (2005): 71–81.

[2] Rosenblatt, J. D., and Y. Benjamini. “Selective Correlations; Not Voodoo.” NeuroImage 103 (December 2014): 401–10. doi:10.1016/j.neuroimage.2014.08.023.

Improve this answer
$\endgroup$
12
  • 1
    $\begingroup$ I don't understand "If you are constructing intervals on all considered parameters, there is no need for correction. Regular intervals will control the FCR". Why would there be no need for correction in this case? (I couldn't find this in the referred article). $\endgroup$
    – nominator
    Commented Apr 26, 2015 at 9:26
  • 1
    $\begingroup$ Also, it's still not clear to me how to select parameters (if R<m), as I have likelihood ratios, not p-values. $\endgroup$
    – nominator
    Commented Apr 26, 2015 at 9:28
  • 1
    $\begingroup$ From [1]: unadjusted CIs seem more acceptable than "... they give the right coverage on average; the proportion of 95% CIs covering their respective parameters out of the intervals constructed (namely, the number covering divided by the number of parameters m) is expected to be .95, and thus only .05 will not be covered..." $\endgroup$
    – JohnRos
    Commented Apr 26, 2015 at 12:12
  • $\begingroup$ Regarding the selection: Are you constructing intervals for all likelihood ratios? If so, there is no selection (R=m). How to construct a marginal interval on the likelihood ratio is not a multiple-testing matter. Just construct the intervals as you would construct for inference on a single parameter. $\endgroup$
    – JohnRos
    Commented Apr 26, 2015 at 12:15
  • 2
    $\begingroup$ Actually the problem I have with this approach is that I want to select parameters based on their significance. The number of parameters to select should not be fixed a priori, and not just based on the likelihood ratios because that would ignore their significance. That's why I prefered the "FCR-Adjusted BH-Selected CIs" procedure. It selects parameters based on their p-values and then constructs 1-Rq/m CIs for them. The only problem I had, as stated in my question, was how to get p-values from likelihood ratios. But I have figured this out now. $\endgroup$
    – nominator
    Commented Apr 30, 2015 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.