2
$\begingroup$

I use glm.nb from MASS package over many observations. I then extract the p-values for the interaction term. Due to multiple testing, I use BH method for correction for multiple testing, with FDR of 0.05. Sometimes, when I apply the negative binomial model to fit to the data, the model doesn't converge and ends up with a warning "Iteration limit reached" and has a huge theta value normally (> 10000). However, most of these times, the p-values obtained are quite small (< 10e-5). Of course these small values are wrong, as there was no convergence achieved. Now, I can find out these observations using the $th.warn attribute which stores the warning message.

My question is, before performing multiple testing, should I remove these non-fit observations and therefore wrong p-values (as they might affect the BH process of correcting for FDR) or because I have already performed the test, removing these entries after finding the warning message would bias the FDR correction of the p-values and therefore I should just continue with FDR correction without removing these obviously wrong but small p-values???

Thank you!

$\endgroup$

1 Answer 1

2
$\begingroup$

I'd speculate that unless the p-values are from converged models, you should drop them before doing the FDR correction. Consider that you are interested in not really globally controlling for the FDR, but more precisely are (only) interested in controlling for the false discovery rate on converged models.

$\endgroup$
1
  • $\begingroup$ Hi drknexus, thank you for your reply. Makes sense. This is what I have done as well. I'll wait a while before marking it as an answer. $\endgroup$
    – Arun
    Feb 24, 2012 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.