2
$\begingroup$

I have obtained EC50 values and their corresponding fiducial limits for dose response curves for several different compounds. I understand that if the fiducial limits are not overlapping for a pairwise comparison, they are different. However, how would one correct for multiple comparisons when there is more than one comparison?

model_result <- drm(Deads/Totals ~ Doses, weights=Totals, data=finney71, 
                    fct=LL2.3u(upper=1), type="binomial")
ED(mod_result, 50, interval="fls")
$\endgroup$
4
  • $\begingroup$ This question is statistical data analysis and should be re-opened. The fact that it is set in a real-world context of drug potencies should not be taken as a reason to close it. $\endgroup$ May 9, 2017 at 20:55
  • 1
    $\begingroup$ This is ambiguous to me, @MichaelLew. I don't think it was closed because it is about drug potencies, but because it looks like it is seeking R code. I'm not sure whether it is or isn't, TBH. $\endgroup$ May 9, 2017 at 23:48
  • 2
    $\begingroup$ The question, "However, how would one correct for multiple comparisons when there is more that one comparison?" can be treated as more open ended than a simple request for R code. See my answer as one possible helpful response that does not include any code. $\endgroup$ May 10, 2017 at 6:38
  • $\begingroup$ Fair enough, @MichaelLew, I'll vote to reopen. $\endgroup$ May 10, 2017 at 15:03

1 Answer 1

1
$\begingroup$

This sounds like a circumstance where "correction" for multiple comparisons is a bad idea.

If the compounds differ chemically then you should expect their affinities and efficacies to differ, and thus differ in their $\text{EC}_{50}$s, even if only by a trivial amount. That means that you know the null hypothesis of zero difference is false. Hypothesis testing in that circumstance is probably not going to give you the type of answer that you want.

Can you define the goals of your analysis? If you want to select the most potent compound then go with the lowest $\text{EC}_{50}$. No test necessary. If you need to decide between compounds on the basis of differences in their potencies then you know more about the size of difference that is important than any statistical test does.

(I will note that non-overlap is a very stringent criterion for 'difference', as non-overlap of two 95% confidence intervals corresponds to a P-value much lower than the 0.05 that might be expected.)

$\endgroup$
7
  • $\begingroup$ Thanks for the comment Michael. I have a list of compounds with LD50 values that I want to compare to say for example that compound x is more toxic than y, with some statistical confidence. Prior literature has suggested there are no differences between these compounds. Since your comment, I have found that the ED() command can give the result and SE in the log('e') form for variable 'e' in the Hill's equation. I can thus run t-tests on these and correct for multiple comparisons with p.adjust(). $\endgroup$ May 9, 2017 at 22:55
  • $\begingroup$ Read my answer again. There is no pharmacological point in generating P-values, and even less point in adjusting them to prevent the increase in the false positive errors that are themselves impossible. I am not universally against adjustments and I don't hold that the null hypothesis is always false. However, in this case there is a theoretical reason to expect that the null hypothesis cannot be true. Don't test it. $\endgroup$ May 10, 2017 at 6:41
  • $\begingroup$ I appreciate you pursuing this with me. These are ten chemically similar compounds (some are enantiomer pairs) and we don't know what to expect. Thus, knowing whether one is more potent than another is a question we'd like to answer, or alternatively a ranking of potencies. LD50 values have uncertainties, so how does one say they are (statistically) different without a test? We originally discussed differences based on non-overlapping fiducial limits, but a reviewer requested formal testing with correction for multiple comparisons. How'd you address this request? Suggested papers to cite? $\endgroup$ May 10, 2017 at 23:36
  • $\begingroup$ If it were my own drug discovery program I would inspect the rank order of potencies. If the potency differences are functionally small then treat them as functionally equivalent. How small a difference is functionally trivial is not the same question as the statistical test if going to answer. The test will tell you which differences are statistically small, which is to say small relative to the observed SEM. You can make the SEM small by having many replicates and large by having few. $\endgroup$ May 10, 2017 at 23:56
  • $\begingroup$ The problem with using a multiplicity ``correction'' in this context is that you can make the drugs not statistically different from each other simply by testing lots of drugs. The differences that matter are those that are pharmacologically relevant, not those that are big enough given your sample size to withstand the power-sapping effect of a multiple comparison procedure. $\endgroup$ May 10, 2017 at 23:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.