Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I have preformed multiple tests and now have a list of all the calculated P-Values (~30K^2 values). I have a reason to believe my tests are somehow dependent.

I would like to analyze the results and correct the p-values.

Unfortunately, I only have very basic knowledge in statistics. From the little I managed to gather, I think the Benjamini Hochberg method for FDR control should be suitable for my needs.

My questions are:

  1. Am I right?

  2. If so, I'd like to implement a code that preforms this calculation. Is the following code correct: http://www.augix.com/wiki/index.php/Benjamini_and_Hochberg_FDR_control and does it calculate the cutoff for my case?

Thanks!!!

share|improve this question
Please don't attach category to the title -- tags are sufficient for that. – mbq Jan 15 '12 at 16:33
I've calculated the critical F-Value according to this: en.wikipedia.org/wiki/False_discovery_rate#Dependent_tests (from the paper: The control of false discovery rate in multiple testing under dependency by Benjamini and Yekutieli, 2001). – l.g Jan 17 '12 at 10:12

1 Answer

BH is reasonable in the sense it takes p-values as input for the procedure. However, it will guarantee FDR control only for certain kinds of dependence. Namely, positive regression dependence on a subset (PRDS). It is unclear from your question if this condition holds. If you believe the variables are pairwise positively correlated, it will do. If you suspect there might be some negative correlations, BH is not the way to go. At the cost of some power, you can do the more general test you mentioned in your comment. If you are using R, then the function p.adjust is all you need. Look for the BY method.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.