1
$\begingroup$

My dataset is 5175 rows by 16 columns. Each row is a biological pathway, each column is a sample (n=4/group), and each observation is an enrichment score. Due to the small sample size and non-normal distribution, I am choosing nonparametric tests.

My goal is to first run a Kruskal-Wallis test between the four groups for each biological pathway and then apply the appropriate post-hoc tests.

My questions are as follows:

  1. What is the appropriate post-hoc test following KW? I have heard both Dunn and Mann-Whitney can work but am not sure
  2. Do I need to apply an FDR correction to the Kruskal-Wallis test due to the fact that I will be running >5000 tests?
  3. For the post-hoc pairwise comparisons, how do I simultaneously adjust for multiple comparisons (i.e. 4 groups) as well as for the fact that I am running >5000 tests?

Example R code:

#run KW test for each row
data$KW.pvalue <- apply(data,1, function(x) {
kruskal.test(values~ind,
           data=stack(data.frame(cbind(
             groupA=as.numeric(x[1:4]),
             groupB=as.numeric(x[5:8]),
             groupC=as.numeric(x[9:12]),
             groupD=as.numeric(x[13:16]))
             )))$p.value})
Improve this question
$\endgroup$
2
  • $\begingroup$ It's better if you don't post a list of clearly distinct questions in the same post. That first question doesn't require anything from the rest of your post and should be asked separately, but there are already a number of questions already on site answering it. Please try a search of our site. e.g. see stats.stackexchange.com/questions/25815/… or stats.stackexchange.com/questions/162679/… $\endgroup$
    – Glen_b
    Jul 26, 2017 at 0:54
  • $\begingroup$ @Glen_b The context here is important I think. The itemized questions are closely related in that, for this project, they interact and need to solved together. The trouble with breaking the question up into separate very specific questions is that it would lose the higher perspective of whether these specific statistical tests are the best way, or even a acceptable way, to achieve the ultimate aims of the project. $\endgroup$
    – Gordon Smyth
    Jul 26, 2017 at 1:03

2 Answers 2

Reset to default
3
$\begingroup$

Are you familiar with the Bioconductor project? I suggest that you have a look at R software packages in that project, because they are specifically designed to address statistical bioinformatics problems such as yours. A lot of statistical methodology has been developed in this field and simply applying univariate tests like kruskal.test() row by row is hardly ever an optimal way to proceed. The Bioconductor support forum is very active and you would be able to get feedback about how to proceed.

You don't say where your enrichment scores are from. Have you perhaps output enrichment scores using ssGSEA from the Broad Institute?

You don't say what the pathways are. Are they perhaps gene sets from the Molecular Signatures Database?

The problems of small samples and of multiple testing over >5000 pathways are far, far more important to deal with than issues of posthoc tests or non-normality. One way to proceed would be to transform the enrichment scores to rough normality (it only needs to be rough) and then use the limma software package to do a differential enrichment analysis. (Note I'm the limma author.) limma conducts t-tests and F-tests and uses empirical Bayes methods to leverage information from all the rows at once. limma offers rich possibilities for FDR control over both multiple rows and multiple comparisons. It is also provides far better statistical power than a univariate test like Kruskal's can give.

Alternatively, you could use limma's pathway analysis functions directly. Limma functions like roast(), camera() or fry() can conduct differential pathway enrichment analyses directly from your raw data, without computing single-sample enrichment scores.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks so much for your response. I have heard of limma but have not used it myself. I apologize for not providing some relevant data. The enrichment scores are output enrichment scores from the ssGSEA module from GenePattern (Broad). The Pathways are the C2.all segment of MSigDB. I have heard of empirical Bayes methods being used on ssGSEA data, so perhaps that will be a good point of contact moving forward. All the best. $\endgroup$
    – Steven
    Jul 27, 2017 at 1:29
1
$\begingroup$

@Gordon_Smyth is correct, I regularly use his package for my works. I wouldn't apply KW test row-by-row in your case unless you have a small number of genes (but you have 5175 genes). I'd let the limma package work out the details.

  1. If you really want to do KW. Dunn's Test is fine. (Post-hoc tests after Kruskal-Wallis: Dunn's test or Bonferroni corrected Mann-Whitney tests?)
  2. Yes. Not applying any correlation for your 5175 genes is a statistical mistake.
  3. You don't really need to adjust for pairwise comparisons.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.