1
$\begingroup$

I am doing wilcoxon-rank-sum tests of the sensitivity (AUC) of over 1000 drugs between mutated gene A and wild-type gene A.

So, for each drug, there will be one wilcoxon test and one p-value. Since I have 1000 drugs, there will be 1000 tests and 1000 p-values. And then, I am putting all the p-values gained from each individual tests (y-axis) into a volcano plot with x-axis being the effect size of AUC between mutated gene A and wild-type gene A.

I am wondering if I should plot the volcano with the p-values adjusted by multiple testing correction like Benjamini-Hochberg (BH) procedure. I found that some paper do this correction, while do not. So, this is making me confused if I should do the correction.

$\endgroup$

1 Answer 1

1
$\begingroup$

The issues of how you display the volcano plot and how you make "significance" claims on the data are different.

Provided that you are clear about which method you use, you can display the volcano plot either way. As the Benjamini-Hochberg procedure controls false discovery rates, not family-wise error rate, if you choose to display the B-H values you can avoid some confusion by reporting them as q-values. If a referee disagrees, it won't be hard to switch to the other display.

That said, you should not make statements about statistical significance on the uncorrected p-values. For this type of study, statements about positive findings are typically made with respect to false-discovery rates.

$\endgroup$
12
  • $\begingroup$ I am thinking each test actually has an independent hypothesis. hypothesis of test 1: drug 1 is more significantly sensitive to mutated gene A and wild-type gene A hypothesis of test 2: drug 2 is more significantly sensitive to mutated gene A and wild-type gene A .... it can come out that several drugs are significantly more sensitive to mutated gene A than wild-type gene A. So, why multiple testing correction should be done to compare the significance between all of these individual hypothesis. $\endgroup$
    – Idiotsum
    Commented Oct 8, 2020 at 10:57
  • $\begingroup$ I have several drugs of the same class that are significantly more sensitive to mutated gene A than wild-type gene A when p-values are uncorrected. But they are no longer significant after BH correction (FDR 5%). so in this case, I cannot report that this class of drug shows higher sensitivity to mutated gene A, right? I am not sure if my thought is correct. Please correct me if I misunderstand it. $\endgroup$
    – Idiotsum
    Commented Oct 8, 2020 at 10:58
  • $\begingroup$ @Idiotsum the multiple comparisons problem is that if you do 1000 tests at p < 0.05, you expect to get about 50 positive results just by chance even if there are no real differences. That's what you need to protect yourself and your audience from by multiple comparison corrections. $\endgroup$
    – EdM
    Commented Oct 8, 2020 at 13:25
  • $\begingroup$ @Idiotsum there might be ways to take into account sensitivities of drug classes with respect to gene mutation status more efficiently, but that would require a different type of analysis. For example, you could use a mixed model, with the drug classes as a fixed effects and the individual drugs as random effects within their corresponding classes. Then your individual hypotheses are about the drug classes rather than the individual drugs, with much less of a multiple-comparisons problem. $\endgroup$
    – EdM
    Commented Oct 8, 2020 at 13:29
  • $\begingroup$ Do u mean I could use a mixed model: drug sensitivity (AUC) ~ class of drug (fixed effect) + gene A mutation status + individual drugs within corresponding classes of drug (random effect) to see if gene A mutation is a significant covariate affecting AUC? $\endgroup$
    – Idiotsum
    Commented Oct 10, 2020 at 8:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.