Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I use glm.nb from MASS package over many observations. I then extract the p-values for the interaction term. Due to multiple testing, I use BH method for correction for multiple testing, with FDR of 0.05. Sometimes, when I apply the negative binomial model to fit to the data, the model doesn't converge and ends up with a warning "Iteration limit reached" and has a huge theta value normally (> 10000). However, most of these times, the p-values obtained are quite small (< 10e-5). Of course these small values are wrong, as there was no convergence achieved. Now, I can find out these observations using the $th.warn attribute which stores the warning message.

My question is, before performing multiple testing, should I remove these non-fit observations and therefore wrong p-values (as they might affect the BH process of correcting for FDR) or because I have already performed the test, removing these entries after finding the warning message would bias the FDR correction of the p-values and therefore I should just continue with FDR correction without removing these obviously wrong but small p-values???

Thank you!

share|improve this question

1 Answer

up vote 2 down vote accepted

I'd speculate that unless the p-values are from converged models, you should drop them before doing the FDR correction. Consider that you are interested in not really globally controlling for the FDR, but more precisely are (only) interested in controlling for the false discovery rate on converged models.

share|improve this answer
Hi drknexus, thank you for your reply. Makes sense. This is what I have done as well. I'll wait a while before marking it as an answer. – Arun Feb 24 '12 at 16:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.