I have a multivariate dataset where each sample has two factor labels and hundreds of genes as counts. Here is a simplified example of what the data look like:
| factor1 | factor2 | gene1 | gene2 |
|---|---|---|---|
| 1 | A | 4 | 20 |
| 1 | B | 10 | 19 |
| 2 | A | 5 | 3 |
| 2 | B | 9 | 2 |
The question I have is:
Which factor classifies the samples better based on their gene composition?
I could imagine a number of ways to try to answer this question.
PERMANOVA A simple approach would be to choose a distance metric and perform a PERMANOVA and see which factor explains the most variance (R2). For example, perhaps both factors would be significant, but factor1 would explain more variance (15%) than factor2 (10%). However, that doesn't feel like a valid approach, because just because both factors are significant doesn't mean they're significantly different from each other.
Hierarchical clustering Another approach would be to use an unsupervised clustering approach such as hierarchical clustering, as discussed in this post, and see which factor splits the tree first. For example, perhaps the first split in the tree would be factor1 1 vs. 2 and underneath 1 and 2 would be factor2 A vs. B. This would indicate factor1 was "more important" than factor2 when discriminating between samples. Or I could use "external cluster validation," but I'm not sure how to do that. However, I don't know if this necessarily demonstrates factor1 is more important or if there's a statistical test to compare the two. Plus, it feels like it is unnecessary to use an unsupervised approach given that I have labels that I'm interested in as discussed in this post.
Classification I could also imagine using a supervised classification test such as random forest or logistic regression. For example, I could train a random forest model with both factors as predictors then rank the importance scores for the two factors based on classification error rates. Alternatively, I could build separate models, one with factor1 and one with factor2 and compare their test set classification error rates. Although, I'm not sure if there's a statistical test to compare two models like this.
I'm not sure if any of these approaches are valid. None of them feel "right" to me, but this problem is a little bit out of my wheelhouse from a statistical standpoint. Any advice on how to answer this question would be helpful. Thanks!
