Question: Which statistical significance test to use?

Question

I am wondering what statistical significance test I can use for the following use case but I cannot find one. The data consists of more than two groups of unequal size where multiple samples in a group can come from the same individual and the outcome variable is non-normally distributed for every group but an individual only appears in one group. Thus, I am looking for a test for dependent groups (within-group dependence) that can ideally handle non-normal data (or it could be transformed) for more than two groups. Any help is greatly appreciated!

I think I can use the Wilcoxon rank sums test for two groups (if the equal variance assumptions holds) but would like a test that accepts more than two groups.

As context: I need to do this analysis for my master thesis and a I seemingly can not find an answer anywhere

Answer 1

You're focusing on normality of the data, which doesn't matter in a robust sample size, and ignoring the correlated data, which does matter. Anyway, Wilcoxon doesn't work on a $k$-way test, $k>2$. You're thinking of Kruskal Wallis in that case. But forget about rank-based statistics for the moment. Correlated data introduce intrinsic issues with testing and 95% CI calculation that don't resolve with moderately large samples. What you probably need is a GEE so you can link repeated measures within groups before performing inference. An exchangeable correlation is a common choice, something called a repeated measure ANOVA.

You can't assess normality without a graphic, and you could do more to describe what you saw. While normality isn't a formal requirement for GEE (or ANOVA, or a t-test), we might guess you encounter a common form - skewed data. Then consider a log transform of the data to a. improve inference and b. improve interpretation of the results. When the $y$ is log transformed, the hypothesis is essentially the same - a 0 mean difference corresponds to a 0 geometric mean difference, but a significant result is described in different terms - if you exponentiate the mean log difference, the value is a percentage difference between response among groups.